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ABSTRACT 



An expert system and processing method employing a three 
level hierardiical knowledge base that has a plurality of 
nodes coupled together. An uppemiost level comprises a 
behavioral knowledge level, a middle level con^irises a 
structural knowledge level and a lowemiost level is an 
action level. Inference processing proceeds from the behav- 
ioral knowledge level through the structural knowledge level 
to a leaf node of the action leveL One or more non* 
inferentially assessable utilities may be associated with the 
inference kernel of the expert system for accessing during 
inferential processing within the hierarchically structured 
knowledge base. Almowledge editor implements guidelines 
that structure received information in the desired hierarchi- 
cal three level configuration. Further, interactive 
mult^media^ypeImedia sy^ems and methods are presented, 
which might be expert system based. lYaining and certifi- 
cation applications of the multimedia/hypermedia system/ 
method are given. 
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EXPERT SYSTEM AND METHOD system shdls allow the encoding of strategic and object- 

EMPLOYING HIERARCHICAL level knowledge as meta-niles. However, this requires 

KNOWLEDGE BASE, AND INTERACTIVE extensive knowledge of the progrannning paradigm and the 

MUUIMEDIAyHYPERMEDIA development environment Another dass of tools provides a 

APPLICATIONS 5 search algorithm for a flat problem-space representation. 

Although the problem rq^resentatioQ is simplified, the 

TECHNICAL FIELD search compleadty for a problem of solution length L, and a 

search space hrandiing factor B has a worst case complexity 

The present Invention relates in a first aspect to expert 0(B^). 

systems, and in particular, to a domain independent expert alternative, it has been suggested that it is actuaUy 

system and method employing inferential processing wifliin ^^^^^ ^ certain domains to concentrate on construct- 

a hicrarchicaUy-structurcd knowledge base and non- models of belief organization for diagnosis rather than 

inferential accessing of information within one or more models of physical entities. The concept of belief potentially 

associated utilities. In another aspect, the invcntionrdates to ^ ^ ^^^^ explicitly defined knowledge. A 

a unique hicrardiical structure for representing infonnation method of oaganization of beliefs has been proposed for 

in a knowledge base for access by an inference kernel and diagnostic problems that provides explicit bdicf organiza- 

to a knowledge editor for facditatmg development of such a ^^^^ ^jj^p^^^ organization of knowledge about physical 

knowledge base by a domain expert In still another aspect, device charadcristics, functionality, and behavior. The 

the invention relates to interactive multimedia/hypermedia ^^^^^^ jg ^^^^^ reasoning about beUef among 

systems and methods, particularly for training and certifi- altematwes, is extensible, and can be scaled to problem size, 

cation applications. Further, it is assatcd that beUef manipulation coupled with 

BACKGROUND ART information about fault history, events, and symptoms is 

sufficient to secure a good diagnostic result. The approach 

QM (con^uter-integrated manufacturing) is viewed as an presented herein modifies and combines behavioral knowl- 

emerging technology in the domain of manufacturing. Ihe ^ edge presentation with structural knowledge presentation to 

concept of QM is that the whole performs better than the identify a recommended action. 

sum of the individual parts. In order to make a system run it has long been recognized that hierarchical problem 

smoothly with minimum delay (Le., continuous flow solving can be used to reduce search space within an cxp&i 

manufacturing) it is necessary to have a diagnostic system system It has been shown that the con^lexity of search 

for discovering any cause of system failure. It is desirable space for a problem of solution length *U and search space 

thatthisdiagnosticsystembecapableof performing its task branching factor *B *, is reduced from 0(B^ for single-level 

as fast as possible since tiic duration of the system's down representation, to 0(VE) B"^ for two-level representation, 

time is very closely related to productivity and cost and to 0(L) for multi-level representation. This analysis is 

Thereby, tha:e is a need for intelligent diagnostic systems. In based on the assumption that the abstraction divides the 

artificial intelligence and expert systems, diagnosis has been problem into 0(L) constant size problems tot can be solved 

given mjM-e attention in recent years, including trouble- without backtracking. It can be shown that even in 

shooting in electronic circuits and medical diagnosis. domains that do not satisfy the assumption, the use of 

Therehavebeenavariety of approaches taken by various hierarchical problem solving still produces significant 

researchers for creating intelligent diagnostic ex|xat sys- reductions in search space. Hierarchical problem solving 

terns. These approaches range from shallow reasoning using ^ and knowledge processing techniques have been used in 

compiled rules to model-based deep reasoning systems that several reported expert systems. The conceits presented 

reason by ^ploiting causal, structural, and functional relar hmin provide enhancements over such prior techniques. 

tionshq>s. Some ^)proaches even combine shallow and deep E3q)erienced engineers, technicians, and system operators 

knowledge reasoning. It has been hypothesized by some that typically develop their own diagnostic flowcharts, proce- 

the use of deep representations of entities to be diagnosed is dures and shortcuts over time. Such materials are utilized 

superior to empirical knowledge about associations between during daily activities, and only occasionally do such indi- 

malfunctioning parts of an entity and symptoms. The ratio- viduals refer to basic princ^les and documented diagnostic 

nale for sudi a conclusion is that one could not cxhuastively procedures to solve problems. This has been a naUiral 

catalog all such associations, and without such a catalog, a response to the large volume of maintenance and operation 

heuristic-based diagnostic system becomes faritde and fails jq manuals offered by tool vendors and process developers of 

when presented with a case that it does not understand. a manufacturing enterprise. Further, flie majority of exper- 

On the oflier hand, deep knowledge representation is tise accumulated by technicians and operators is usually not 

based on models fliat are difficult to construct, especially shared among different teams and can be lost due to transfers 

models that exhibit the technology intent of the designer. or job changes. In addition, local upgrades/modifications, 

Further, it is unlikely that models will mirror the future 55 which might not be e:q)lained in a vendor manual, can be 

txhavior of entities of any complexity, particularly with scattered among personal notd)ooks or other notes near the 

regard to providing infonnation about multiple perspectives. machines or technician workbenches. The challenge of 

Moreover, modds are likely to be domain specific and oidy knowledge engineering in such an environment is to acoom- 

some fraction of knowledge will be trartsferable from one modate various sources of expertise and to guide and 

diagnostic system's knowledge base to another. go influence the expert toward considering aU aspects of the 

It is also recognized that rule-based systems become environment beyond the individual's usual activities and 

increasingly diffiailt to understand and mninfflin as the concerns. 

number of rules grow. While a reasonable rule-based expat The knowledge acquisition task is often coixq)licated by 

system shell can assist a domain e:q>eit in formulating the fact that human experts have not analyzed the contents 

cause-and- effect rules, a collection of such rules typically 65 of their thoughts, so that they are not explicitly aware of die 

wiU not function as an cxpctt system, except for the most structure of their knowledge. As a result, the iatennediate 

simple cases. To overcome these linutations, some expert steps and the reasoning seems otmous to them and they 
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cannot dearly provide an ovexaH accoont of how decisions Conq>uter-based training and intelligent tutoring systems 

are made, at least not at a level of detail required for expert are typically based on a single, and rather 5inq)le user modeL 

system development There are a number of approaches to Regardless of the familiarity or lack of familiarity of the 

laiowledge acquisition. Tlie three basic approaches are trainee with the topic, all trainees go throug^i the same 

interview, interaction (supervised) and induction 5 training process. A new employee who has never worked in 

(unsupervised). a similar environment is given the same training material as 

Historically, interview has been the most prevalent an employee who has many years of related experience and 

method of knowledge acquisition. It is, however, higjily might have been transferred to a new assignment from a 

dependent on the knowledge engineer, is often time coo- similar area. Certification and qualification procedures are 

suming and expensive, and is typically viewed as tiieknowl- lo usually conducted OTally in an ad-hoc feshion and arc 

edge acquisition bottleneck. It is also recognized that for subject to a trainer's judgment and biases. In addition, there 

knowledge intensive and task oriented applications, super- is no formal methodology for increasing the responsibility of 

vised knowledge acquisition is more efficient Large data- a trainee as the training paroceeds. In order for a conD5)uter- 

base and data intensive applications are good candidates for training system to acquire the necessary flexibility, it 

unsupervised knowledge acquisition. ^ should distinguish between several types of students and 

On the othtr hand, many expert systems use task-oriented "^"^ '"^^<^ * '"^ 

shallow knowledge. In knowledge acquisition it is some- backgrouna. 

times necessary to carry out excursions into deep knowledge Multimedia in general, and motion video, animation and 

in ordCT to understand and validate associated shallow audio in particular, are beUeved very appealing to personnel 

knowledge. This requires involving experts in knowledge ^ involved in maintenance, diagnostics and training applica- 

acquisition, since they arc in a position to understand the ^ons. However, in many cases a major drawback is the cost 

deep knowledge and its relation to the shallow knowledge. and logistics of hardware and software required to imple- 

In addition, studies performed on the quality, efficiency, and ment such a multimedia system. This is conq)licated by the 

accuracy of knowledge bases have shown that knowledge manufacturing facilities include tens if not 

bases developed by the expert (versus the knowledge ^ hundreds of workstations. If an application requires special 

engineer) tend to be smaUer, provide larger number of hardware and/or software Hcensing, then cost becomes sig- 

patfaways associating evidence to diagnostic hypothesis, nificant 

include more critical attributes, and provide richer dusters All the above drawbacks and problems assodatcd with 

of knowledge. This is another argument in favor of providr existing expert systems and multimedia/hypermedia appli- 

ing domain experts with a larger role in knowledge acqui- cations are addressed by the systems and methods presented 

sition. herein. 

As a related i^Wem, existing multimedia training appU- dkcl OSURE OF INVENTION 

cations are typically developed by a media production DISCLOSURE OF INVENTION 

specialist, and are usually delivered on read-only storage Briefly described, the present invention comprises in a 

medium (CD-ROM, video disk). Such applications exhibit first aspect an e^^ert system responsive to provided external 

huge initial start-up costs, do not provide any intelligent information. The expert system indudcs a first storage 

feedback or control, and require specialized hardware and/or means and a second storage means. The first storage means 

software, meaning that any modification and/or upgrade will stores a hiCTarchically-structured knowledge base containing 

be expensive and time consuming. ^ inferentially processable information, while the second stor- 

Manufacturing enterprises can enq)loy hundreds and age means contains at least one utiUly comprising non- 
somctimes thousands of permanent and temporary system inferentially accessible information. A processor is coupled 
operators and technicians. Currently, a multitude of methods to the first storage means and the second storage means for 
are used for educating and assisting such manufacturing processing the hierarchically-structured knowledge base and 
personnel These methods include informal, unstructured 45 accessing the at least one utility, respectivdy. The process- 
training sessions, printed *in-house' manuals, classroom ing means indudes means for inferential processing of 
instruction, and walk-through orientations to name but a information in the hierarchically-structured knowledge base 
few. In the maruifacturing environment, the student popu- and means for non-infcrcntial accessing of information in 
lation grows with every group of new employees, whedier the at least one utility. The inferential processing means and 
temporary or permanent This student group is typically 59 the non-inferential accessing means are responsive to the 
large and decentralized and caimot be released from the provided external information. 

manufacturing line simultaneously to attend a standardized In another aspect, the invention comprises a 

training course. hierarchically-structured knowledge base for an expert sys- 

Intcractive multimedia training and certification can tenL The knowledge base comprises a hierarchical decom- 

present consistent subjed matter, on a flexible twenty-four 55 position having a plurality of nodes coupled together in three 

hour, seven day per week schedule. The subject matter distinct levels. An uppermost level con^irises a bdiavioral 

delivered to the trainee is guaranteed to be consistent, thus knowledge levd, a middle level comprises a structural 

avoiding reliance on a thorough presentarion of material by knowledge level and a lowermost levd is an action IcveL 

a knowledgeable technician. Use of an on-line computer The adion levd is implemented as at least one decision tree, 

teased training methodology would eliminate a back-levd 60 The three distinct levels are interconnected such that infer- 

problcm-Currently, with die use of printed documents there ence processing can occur through the hierarchically- 

is always an uncertainty as to whether the line operator is structured knowledge base from the behavioral knowledge 

using the most current version or whether an operator's level through the structural knowledge levd to the action 

training in fact covers a latest version of a processor tool level 

upgrade. By making the inf onnation available on-line, posi- 65 In yet another aspect, an intelligent system is presented 

tive control over the information being disseminated is responsive to provided external information. The system 

obtained. indudes memory means for storing multimedia displayable 
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information, which is noD-iofercntially accessible. An intd- In a diagnostic implcnicDtation, knowledge processing 

ligent control system, coupled to the memory means, provides us^iil information on the nature of a problem, even 

indudcs a hicrarchicaily-stnicturcd knowledge base con- ifdie exact pFohl£m(s) remains unidentified. The knowledge 

taining inferentially prooessable information and processing engineering methodology is based on supervised knowledge 
means coupled to the hierarchically-structured knowledge $ acquisition principles and provides a complete knowledge 

base. The processing means indudes means for inferential engineering strategy directed towards a domain cxpal 

processing of information in the hierarchically-structured Acquisition guidelines and a Knowledge Editor provide a 

knowledge base and means for non-inferential accessing cf ^^^gj integrated solution for reducing the knowledge engi- 

the multimedia displayaWe information in tiie memOTy neering bottleneck, and improving the quality of &e final 

means. The means for non-inferential accessing of multi- Development and deployment of an interactive 

media displayable information IS responsive to the means for diagnostic expert system is possible on standard 

infmntial processmg of infonnation and the means fa: workstations, thereby providing a low cost delivery 

inferential processing of inf<Emation is itself responsive to platform, as weU as enabling the integration of multimedia 

the provided external information. capabilities of graphical workstations into the diagnostic 

In a further aspect, a computer system is provided having applications, 
memory means for staring multimedia display^le informa- ^ presentedls a complete and integrated methodology 
tion and display means for displaymg mformation to a user architecture for development of intelligent interactive 
of the computer systent Aprocessmg means, coupled to the ninltimediamypermedia computer-aided training^edncation 
memory means and the display means, indudcs h^er- qualification/certification appHcations. A control and 
graphic processmg capahOittes associated with a predefined ^ j^j^j^^^ej^e^t knowledge base serves as an intelligent tutor- 
graphics image displayable on the display means. The ^ framework. TMning. certification, and on-line 
hypergraphicprwi^sing capabilities mdude means forpr<v documentation databases are integrated with theknowledge 
vidingtheuserofthecomputersystemwithanoptionsmenu ^^^^ management ^Ucations developed 
upon user selection of flie predefined graphics image when ^ methodology and architecture reside entirely on 
dispkyed by the display means. The options ma^^ ^ hard disk or removable/rewrilable storage medium (such as 
least two user selectable options, eadi of vAach is Unked to storage), and can be used in stand-alone or nrtworic 
a separate portion of the multimedia displayable information configuration 

stored in the mcmoiy means. Each separate portion of the ^ * 

multimedia displayable infonnation presents inforaiation to BRIEF DESCRIPnON OF DRAWINGS 

the usa: in a different multimedia format. 30 j^^^^ ^ther objects, advantages and features of the 

In a still further aspect, a method for defining a present invention will be more readily understood from the 

hierardiically-stractured knowledge base for an intelligent following detailed description of certain preferred embodi- 

systan is presented. The method includes the steps of: ments of the invention, when considered in conjunction with 

creating multiple knowledge trees from a plurality of nodes accoiiq)anying drawings in which: 
andassodatedattributcsincludingatleastoneactionknowl- 35 pKJS. 1^, lb & Ic are examples depicting the hierarchical 

edge tree, at least one structural knowledge tree, and a ^ structure of a knowledge base in accordance with the 

behavioral knowledge tree; and linking the knowledge trees ^^^^^^ invention, wherein FIG. la depicts universal 

together to form a hierarchical decomposition such tiiat the attributes, FIG. lb shows global attributes and HG. Ic is an 

hierarducally-sttucturcd knowledge base has three distinct example of a Condition Action Tree (CAT); 
l(n^ch, an uppermost levdcoH^^ 40 PKJS. 2c, 2& & 2c depict examples of diagnosis pursuant 

edgetcee, a nuddle levelc(Hnpriskg the at te^^ one struct to the present invention using^^us condition action t^^^^^ 

tuialknowledgetice^andaloweimostk^^ processing embodiment for defining a 

at least one ac^on knowledge tree. ^ 3^ ^ ^^^^ 

Also presented is a processmg method for an expert invention; 
systemresponswetoexternalinfo^ « FIG. 4 depicts one embodhnent of a mam menu screen of 

of the ^^stem. Ihe processing metood includes the s^s of : knowledge Editor (KE) developed in accordance with the 

iitfcientiaUy processmg urfonnation m a hi^archicaUy- ^ i^^f ^^n to guide a userl partiti^^^ 

structured knowledge base interactively with the external .. ^. , , 5 i. 

Moimation inputted by the user of the expert system; and a <^osUc knowledge base; 

during the inf«ential processing, non-inferentiaUy access- jo , ^9' " f^P'' f l^^^^^f '""^5 

ing itrformation in at least one predefined utility separate faoHtafBg the defflmtoon of nod^ and th^ attributes, and 

from the hieiarchicaUy-stnictured knowledge base. Addi- ^ow nodes may be hnked together to define a system in 

tionalmethodsandmelhodenhancemcntsarealsoprovided accordance with the present invention, wherein both 

1^^*^ template-based and visual programmmg capabilitLes of the 

To restate, provided herein is a domain independent 55 ^^^^^ shown, , ^ , , ^ 

expertsystemandprocesslngmethodemployinginferential HG. 6 is an example of a Imowledge representation 

processing within a hierarchically-structured knowledge ^<^^y P"^^^ to the present mvcntion; 

base in conjunction wifli non-inferential accessing of infor- PIG- 7 is another example of a flirce level hicrardiy in 

mation within one or more associated utilities. Knowledge accordance with the invention, wherem structural level 

representation architecture is based on hierarchical knowl- 60 knowledge bases are linked to terminal nodes of a behav- 

cdge representation princq)les with a novel decision tree ioral level knowledge base, and condition action trees are 

architecture consistent with a proposed knowledge engineer- ^^^^ ^ terminal nodes of the structural level; 

ing methodology. Conflict resolution employs an entropic FIG. 8 depicts representation of events as timc-stanqjcd 

based approach wbidi allows domain experts to describe the brandies of corresponding condition action trees; 

heuristics practiced in resolving conflicts and unanticipated 65 FIG. 9 is an example of a condition action tree (CAT) 

situations, taking into consideration the practicality and showing a root node, an mtry node and various diildren 

overall cost of the solution. nodes connected thereto; 
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FIG. 10 is a flowchart of one embodinieat of inferential observable elements, factors, or circumstances or identify- 

toowledge base processing by an inference kernel in accor- ing the proper association between symptoms and their 

dance with the present invention; causes. In other words, diagnosis is the process of finding the 

FIG. 11 is aperspectivc view of a computer system upon ^o^^ion and natoe of a system nialfunction and even 

which an expm s^^ and/or a multimediamypermedia 5 suggesting a iei«^. A system is defined as a comply of 

wi^ ^ / jiT^ mtcractmg objects that have some process bdiavion The 

system in accordance with the present invention may be ^Dowing are Examples of systems: a VLSI chip; a computer 

implemented, program; a manufacturing tool or machine (e.g., wafer 

FIG. 12 is a block diagram representation of possible polisher, oxide growth furnace, etc.); a manufacturing 

component overlaps of both an expert system and a group; and a human. 

multimedia/hypermedia system in accordance with the gy ^^y of example, a semiconductor manufacturing 

present invention; process will be discussed herein. In a semiconductor f abri- 

FIG. 13 is an example of one embodiment of a graphical cation environment, a '^malfunction" can be an error in a 

user interface window in accordance witii thepresait inven- computer program, a defect in a VLSI chip, an incorrect 

tion; specificatioDof a manufacturing recipe, a failure of a manu- 

FIG. 14 charts IBM*s common user access (CUA) guide- ^ factoring tool to function properly, etc A "symptom" of a 

lines and user interface design principles en[q)loyed in a malfunction is observed when the system behaves in a way 

graphical user interface in accordance with the present that is not expected. In other words, a symptom is a 

invention* discrepancy between observed, or actual behavior, and an 

FIG. 15 depicts one embodiment of a graphical user ^ expected, or predicted system behavior. A symptom is an 

interface window partitioning in accordance with the present observable condition which indicates an abnormal behavior 

invention* * possible malfunction. Hence, when a symptom exists, 

FIGS. 16a-l6p depict standardized programmable tem- be a condition (fault) which causes the s^onptonL 

riuo. AW xvy ^F^^ owiuumiu*^ yLv^oLiuuaui.^ v^iu "condition** is an underiyinE. invisible, real problem. A 

plates (refened to as 'logical pages") proposed for use m a ^ «Jiiumuu i:* ou uuiiuiiymg, uivi^xuit-, *«u yiuui ui. 

, . ^. &*~r—o r ^ symptom on the Other hand, IS a Visible manifestation of one 

multunedia/hypermedia system in accordance with the 25 " . " ^ 

^ . ^ or more condiUonfs), i.e., evidence of a condiUon(s), such as 

present mvention; , , t^. ^ _* V* 

, , . . ... * a tool malfunction or a system error message. Diagnosis 

FIG. 17 IS an example of a logical page partiUomng of a therefore means identifying the conditions after observing or 

multimedia/hypermedia module cnq)loying a tanplate of indirectly detecting the symptoms. An ^Initiative" is an 

FIGS. ldfl-16p; action or a set of actions necessary to recover from a 

FIG, 18 is an image useful in explaining hypergraphic hot 30 malfunction, or a set of actions (operations) that will trans- 
spot functions in accordance with the present invention; f^^j^ ^ system to a desired state. 

FIG. 19 is one embodiment of a logical page attribute As notedinitially, experienced engineers, technicians, and 

menu useful in customizing logical page templates in accor- operators usually dcydop their own diagnostic flowcharts, 

dance with tiie present invention; procedures, and shortcuts over the years. They utilize these 

FIG. 20 is a block diagram representation of one embodi- during their daily activities and only occasionally refer to 

ment of a knowledge-based multimedia training/certification basic principles or documented diagnostic procedures to 

system in accordance with the present invention; solve problems. This has been a natural response to the large 

FIG. 21 is a block diagram representation of a multilevel volume of maintenance and operation manuals offered by 

training/certification user model in accordance with a mul- ^ tool vendors and process developers of a manufacturing 

timedia system pursuant to the present invention; enterprise. The majority of the expertise accumulated by 

FIG. 22 is a display image representation useful in technicians and operators is usually not shared among 

describing a hot spot pop-up menu for a hypergraphics different teams and can be lost due to transfers or other job 

iniplementation in accordance with the present invention; changes. In addition, local upgrades and modifications 

FIG. 23 is a flowchart of one embodiment of processing 45 (which do not end up in the vendor manuals) can be 

to create a hypergraphic hot spot in accordance with the scattered among personal notebooks, notes on the machines, 

present invention; technicians' workbenches. TTic challenge of knowledge 

HG. 24 is a flwdiart of one embodiment of processing engineering in sudi an environment is to accommodate 

to create a hypergraphic function pop-up menu in accor- various sources of expertise and to guide and influence the 

dance with the present invention; 50 ^=^P^ considering all aspects of the environment, 

HG. 25 is one embodiment of a main menu of a **poor including tiiosc beyond tiie individual's activities and con- 
man's video" (PMV) editorfor authoring motion video clips f«ns This is necessary for eUciting complete and accurate 
in accordance with the piesent invention; knowledge and associated rcasomng strategies. 

FIG. 26 graphically depicts PMV processing in both . There are a number of approaches to knowledge acqm- 

run-time and compile-time in accordance with the present 55 However, the tiiree basic approaches are: 

invention; 1) Interviewing; 

na 27 is a flowchart of one onbodiment of compile 2) Supervised interaction; and 

processing to create a PMV library block in accordance with 3) Unsupervised induction. 

the present invention; and Historically, interviewing has been the most prevalent 

HG. 28 is a flowchart of one embodiment of player «> i^cthod of knowledge acquisition. It is however highly 

processing for viewing a PMV in accordance witii the dependent on the knowledge engineer, and is often time 

present invention. consuming and e^ensive. It is also recognized that for 

knowledge-intensive and task-oriented applications, knowl- 

BEST MODE FOR CARRYING OUT THE g^ge acquisition by supervised induction is more ^dent 

JN^'^ENnON ^ than interviewing or unsupervised induction. Large database 

Various terms used herein will first be explained. "Diag- and data-intensive applications are good candidates for 

nosis" is defined as inferring system maffiinctions from unsupervised knowledge acquisition. 
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The constiuctioa of a knorwledge base is cuncntly a andmoicrespoiisibOityfordevdopmcntandiDaiatc!^ 

bottleneck in the development of AI systems. This is because the expert system. Eadi of these steps is described in detail 

it usually involves interviews with an expert, analysis of the below. 

interview records and protocols, and then fonzmlation of the System development begins by meeting with the 

decision rules. Thus, tiie knowledge acquisition problem is 5 technicians, operators, and managers responsible for ihe 

time-consuming and does not always guarantee the validity tool, process, or task bdng considered for expat system 

of the rules obtained. An important method to overcome in5)lMnentation. Each group is asfced to list what they 

these problems is to assploy inductive learning from perceive to be the problems, time consuming tasks, and 

exanq»les. With this approach, the expert is asked to provide shoBrtcomings of their interaction and involvement with the 

good examples and counter exaiiq)les of decisions, ra&er xo tool or process, and why they perceive each to be a problem, 

than to formalize his knowledge in terms of decision rules. shortcoming, or waste of time. 

These e]ianq)les are then processed by Ac knowledge engi- The above groups are flien asked to explain how they 

neer or by a computer to produce decision rules. presently handle and manage the described problems. To 

The knowledge acquisition task is often complicated by gain insight into the nature of the problem(s) and to help 

the fact that human expats have not analyzed the contents xs explain all aspects of the problem(s) and the environment, 

of their thoughts, so they are not explicitly aware of the the following questions can serve as an outline for flie 

structure of tiieir knowledge. As a result, the intermediate discussion: 

steps in their reasoning seem obvious to them but they "Whst are the short cuts*^ 

cannot deaiiy provide an overall accoum of how their ^ problems? 

decisions are made, at least not at the level of detau required ^ „ ^ , . ^ 0 / .j 

for expat systan development ^ow ttiey would like to handle the problem? (provide a 

On the otha hand, many expert systems typically use 

task-oriented shaUow knowledge. Notwithstanding this, in How would they change their work environment if they 

knowledge acquisition it is sometimes necessary to carry out necessary resources? 

excursions into deep knowledge in orda to understand and 23 Why would they make such changes and modifications? 

validate the associated shallow knowledge. This tradition- What is the return on investment for their organization? 

ally requires involving experts in knowledge acquisition, what is the effect of changes on the product quality, cost, 

since they are in a position to undastand the deq» knowl- and tum-around-time? 

edge and its relation to shallow knowledge. In addition, xhe next step is to follow the technicians or operators for 

studies performed on the quality, efficiency, and accuracy of 3Q a few hours a day, for a few days, and just watch them do 

knowledge bases have shown that knowledge bases devel- their jobs. This is very he^ful in understanding the com- 

oped by a domain expert (ratha tiian by a knowledge plexity and logistics of their jobs and the different sources of 

engineer) tend to be smalla, provide a larga numba of data they need to access. Many cases involve a lot of 

pathways linking evidence to diagnostic hypothesis, include walking around the manufacturing floor to review manuals, 

more critical attributes, and provide richaclustas of knowl- 35 to check the maintenance log, or to consult with odia 

edge. This is anotha argument in favor of providing e;q>erts technicdans for hc^ and suggestions. In some cases main- 

with a larger role in knowledge acquisition. tenance logs have to be checked to detomine the last time 

Considering the above difficulties, the nature of applica- a oon^>onent was changed, a specific service performed on 

tions targeted by the present metiiodology, and an overall a tool, or a process was modified, 

goal of enabling the domain expert to develop the diagnostic 40 The next step is to dosely observe the experts as they 

^plication, a supervised knowledge acquisition methodol- perform diagnosis. They should be asked to explain and talk 

ogy geared toward domain experts is proposed herein. Hiis about what they are doing as they go through the diagnosis, 

methodology is comprised of two stages, namely: xhis involves giving the cjipat a problem to solve and 

A) Con:q>ile, organize, partition, and document die knowl- letting bim/ha talk out loud as the problem is solved. The 
edge and associated reasoning processes according to 43 expat may not fed comfortable with iMs process at first, but 
predefined guidelines (explained below); and it can be explained that you just want to learn how a problem 

B) Use the Knowledge Editor software tool (presented is approached, and that will enable you to understand their 
below) for knowledge entry and verification. style and capture thcdr expertise. As the individual continues 

The two stages of the present ^i^oacfa are acconqilished with ffie diagnosis, their comments, explanations, and jus- 

in four steps. The first step is educating and familiarizing the so tifications should be documented. Comments such as *T am 

potential user with the capabilities and limitations of the going to check the vacuum pressure,** *T just want to make 

knowledge-based system (KBS) and multimedia technology sure there is no vacuum before I test the vacuum switch,** 

(discussed below). This step includes establishing a two- 'T'd betta turn off the main valve before I check the wata 

way communication channd. In die second step, the usa filta," or *1 have to be carcfiil when checking these 

environment is analyzed to identify the potential areas and 55 coimections— th^ are high voltage" are very he^ful in 

tasks suitable for KBS implementatioa In the third step, capturing the structure of the expert's trout>leshooting 

basic principles of developing and utilizing die KBS and methodology, diagnostic knowledge, and reasoning process, 

multimedia technology pertaining to users* domains are The following guidelines and procedures, in conjunction 

described and documented as a set of easy-to-follow guide- with the Knowledge Editor, scnre to induce and capture 

lines and procedures. These guidelines and procedures are 60 complete and accurate knowledge of tiie reasoning strategy, 

focused on problem identification, partitioning, and the heuristics involved in the process. These guidelines 

crganization, and documentation, and are considered an cova tiiree in^rtant aspects of knowledge engineoing: 

integral part of the architecture. They provide a supervised knowledge identification, documentation, and refinement In 

knowledge acquisition methodology consistent with the particular, guidelines 1-9 deal widi identifying various 

proposed knowledge representation and diagnostic reason- 65 knowledge modules and the relationships among these mod- 

ing architecture in accordance with the present invention. ules. Guidelines 10-19 address efficient and complete docu- 

And finally, the usa is gradually encouraged to take on more mentation of knowledge, while guidelines 20-27 concern 
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refining the knowledge id^tified and documented in the 15) ^^te comments, extra instructions, warnings, things to 

previous two stages for mapping to the Fqn-esentatioD archi- look for, excqstions, etc, outside of the node, 

tecture and inference algorithm suitable for interactive diag- 16) On arcs connecting a node to its children nodes, write 

nosis in accordance with the present invention. There are down the condition under which the hrandi is valid (e.g. 

several techniques available to knowledge engineers for 5 Yes, No, >100, <=Max Spec limit, etc.). These coire- 

improving validation and verification. Articles in the avail- spond to possible responses to the questions being asked 

able literature discuss exhaustive testing, case-based testing, or measuronents made, etc. 

formal specification, functional programming, critical 17) Assign a node strength (NS) to each node, representing 

testing, mutation testing, and reliability. Finally, the knowl- your belief that the node would be the proper diagnosis 

edge validation methodology described in steps 28-30 is a lO (option, syn[q)tom) fcr the parent node. Let INS=I for all 

combination of exhaustive, case-based, and critical testing children of a node. 

methods. 18) Assign a node cost (NC) to each node, representing the 

Identify Knowledge Modules '^""^ . i^^^' manpower, resources) of 

^ pofoiming, tcstmg, or vcrifymg the conditions, tests, and 

1) Identify and list all tasks involved. 15 actions described at the node (O^NC^l). 

2) Identify and list all sub-tasks in each task. Tasks should 19) For each node, determine the most useful and meaning- 
be partitioned at a point where either there is a pause in fill order for sorting and presenting all its children nodes, 
the scenario or the nature of the procedure changes. In (Node Rank, NR=1,23, . . . , 99.) 

addition, try to identifv meaningful partitioning criteria „ - „ , . j , 

related to the task, tool, or the environment 20 Refine Knowledge Modules 

3) List features common to all tasks and sub-tasks 20) Identify the source of answers or data for each node. 
(Universal attributes) (FIG. la). Should the user be prompted for the answer, should the 

4) List unique features of each task and sub-task (Global data be extracted £rom a file, or should a database be 
attributes). queried? 

5) list all difi'crcnt ways a task or sub-task can be identified, 25 21) Can the value of a node attribute change during diag- 
scheduled, or performed, nosis or is it fixed once assigned? For example, if **tool_ 

6) List all known causes of or reasons for a task or a id" is fixed, there is no need to ask for its value every time 
sub-task. it is encountered during the diagnosis. Whereas, 

7) Make an ordered list of the steps involved in each task and "pressure," *'temperature," or 'Voltage** can change and, 
sub-task. 30 therefore, the most rec^t values should be obtained each 

8) If a sub-task includes more than 20 stq)s, see if it can be time encountered during the diagnosis. 

partitioned. 22) What type of data should be expected for a question 

9) Generate a high level flowchart of the tasks and sub-tasks (character, intega, real)? 

(FIG. Ih). 23) How should the question be presented to the user? 

Document Knowledge Modules " ^^"^^ be i«esented wilh toe question and an 

* ordered hst of possible answers to select fi-om? Should the 

10) Generate a Condition Action Tree (CAT) (discussed user be presented with a free format data entry panel? 
below) for each sub-task (HG. Ic). Use solid arcs to Should the user be told what is the valid range or type of 
represent strong links and dotted arcs for weak links answer? 

(HGS. 2a~2c). Weak links can represent "maybe,** 40 24) Clarify the questions, actions, and measurements asso- 

"possible,** "something to keep an eye on " or **the last dated with each node in more detail, so a person not 

recourse, in case all other options faH.** familiar with the envirormient can understand (WHAT). 

11) Name the root node of each CAT according to the These textual descriptions will be available to users as an 
problem which it represents. The root node is where all e:q)lanation clarification of prompts and system dia- 
universal and global attributes are defined. These 45 logues. 

attributes are valid throughout the diagnosis (universal) or 25) Describe the reasons , importance, and relevance of eadi 

throughout a problem (global). Local attributes (valid for node to its parent node (WHY). Whenever flie user 

a node) are defined at the individual nodes. In addition, all requests an explanation of and reason for an action 

initial preparations and checkings are done at the root recommended by the system, the system will present the 

The root node leads to a single node (entry node), as 50 information associated with the current node's WHY 

shown in FIG. Ic. This is the beginning of a task or a attribute. 

sub-task CAT. If there is recursion in the process, it 26) Compile a list of services required at each step during 

returns to this node, not the root node (see FIGS. 2a-2c). the diagnosis (e.g., display images, play audio or 

12) Name each sub-task after its parent task. For example, animation, show slide show, present textual or graphics 
if the task is "robotic arm movement problems,** then 55 information, etc). 

sub-tasks could be "aim does not move up,*^ "aim does 27) Enter the data compiled in previous steps via the 

not move clockwise," "arm does not move counter Knowledge Editor. Hrst define nodes, then CAFs, then 

clockwise,** "arm does not stq),*' etc. problems, and finally the system bdiavior and the prob- 

13) Complete and eJtpand the CAT by writing down the lem hiaarcfay. Assign uso* prompt, WHAT, WHY, NS, 
condition(s) to be tested at each node, the question(s) to 60 NC, NR, global and universal attributes, input source(s), 
be asked* the action(s) to be performed, and service(s) to and services (e.g., show pictures, play video, motion or 
be provided. Each node represents a decision, an animation, generate and present charts, ^) for all nodes, 
inspection, an action, or a measurement. _ v h- 

14) Give eadi node a meaningful name which indicates the Knowledge VcnUca&on 

actions carried out at that node. For example, '*is_ 65 28) Verify the knowledge bases and export to a general 

vacuum_ok?,** or "chlLJiouse_air_pressure,** This will purpose classification or decision network expert system 

facilitate knowledge base development and verification. shell. Knowledge modules are checked for syntax 
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vioktioiis, conflicts* and dashes via a Knowledge Editor 
verification utility, discussed below. 

29) Simulate the ^plication one task at a time. If a problem 
is discovered, rq)eat the above steps, export, and simulate 
to verify. The diagnostic application and the knowledge 
bases can be evaluated through ofiBine simulation and 
interactive simulation. In the offline simulation, the diag- 
nostic episodes are randomly genmted and entered into 
the system in the batdi mode. The ou^ut is either 
evaluated manually ch^ stored in a database for analysis. In 
the interactive mode, the knowledge engineer, the end^ 
users and the expats run sample diagnostic cases and 
evaluate the system recommendations for accuracy and 
completeness. The interactive simulation should also 
include new users performing a set of diagnostic scenarios 
both the traditional way (refening to manuals, calling 
experts or maintenance technicians, etc.) and using die 
developed system, and then comparing the two 
approaches. 

30) Finally, simulate the entire process and field test the 
application. Held testing the system should also be 
viewed as a stage in the system verification and testing. A 
typical system will be in field test mode for several 
mondis before becoming an integral part of a manufac- 
turing process. 

To enforce the abovc-desaibed guidelines and to facili- 
tate conq)iling and developing knowledge bases, a Knowl- 
edge Editor is provided in accordance with the present 
invention. The Knowledge Editor (KE) is developed in a 
suitable environment (c.g., IBM's OS/2 operating system) 
and guides the user in partitioning and defining diagnostic 
knowledge (FIG. 4). Jt provides panels for the user to define 
nodes and all the corresponding attributes (questions, 
instructions, actions, services, NS, NC, NR). CATs are 
defined by linking nodes (see HG. 5), and then combined to 
define problems, and finally problems are linked to define 
the knowledge base. The Knowledge Editor pof onus veri- 
fication and syntax checks. 

One processing embodiment cn4>lpying a Knowledge 
Editor in accordance with the present invention is set f ordi 
in no. 3. Initially, 10 "Start," a hierarchical tree structure is 
begun with definidon of a root node, 12 **Define Root 
Node." The type of tree to be defined is tiien determined, 14 
•TVpc Of TYcc?" If at die behavioral Icvd (DHKD), then all 
universal attributes of the knowledge base are d^ed, 16 
Ttefine All Universal Attributes At Root Node while at the 
structural level (SHKB) global attributes are d^ed, 18 
'T)cfinc All Global Attributes At Root Node.** Odierwisc, a 
Condition Action Tree is defined, 20 **Define Local 
Attributes At Root Node." Once the type of tree is 
established, and attributes have been assigned to nodes, then 
a session transcr^t file is defined and a time stan^ is 
inserted at the top of the file, 22 "Define Session Transcript 
File And Insert Time Stamp At Tq) Of Hie." 

Next, an entiy node leading from die root node is defined, 
24 "Define Entry Node Leading From Root Node." This is 
followed fay definition of all branches and nodes until the 
tree is complete, 26 ^"Define All Branches And Nodes Until 
Tlree Is Completed; Assign Names To Nodes." Preferably, 
eadi node is assigned a meaningful node name and a short 
node ID. The resultant hierarchical tree structure is then 
tested fOT error, 28 *Test Tree For &ror(s)," which is 
accon:q>amed by error monitoring, 30 ''Error?" If arts does 
occur, the source of the error is identified and corrected, 32 
*Tdeatify Source Of Error And Correct" Once testing fails 
to establish txe existence of an error, the residtant tree is 
saved in memory, 34 "Save Tree," and processing 
terminates, 36 lEnd," 
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Hie Knowledge Editor (KE) provides gr^hical as well as 
template-based knowledge entry. Graphical knowledge 
entry (shown in the lower left comer of FIG. 5) provides a 
visual programming envirorunent for defining knowledge 

5 bases. The template-based qiproach (shown in die upper 
right comer of FIG. 5) mi^ be more suitable for developers 
with a mainframe background who feel more comforteble 
with text-based application development environments. 
Grq)hical knowledge entry, on the other hand, is suitable for 
users who are fam^^ with graphical workstations and fed 
comfortable with using a mouse. In either case, the KE 
preferably provides a seamless interface and enables an 
application develox>er to move back and forth between these 
two development environments. 
As briefly noted, there have been a variety of approaches 

13 taken by various researchers to attempt to understand meth- 
ods for creating intelligent diagnostic systems. These range 
from rule-based ("shallow") systems consisting of IF-THEN 
production rules that are highly domain-spedfic, to model- 
based ("deep") systems that reason by exploiting causal, 

20 structural, and functional relationships. Diagnosis in shallow 
systems is fast if the symptom has been experienced and thus 
has been induded in the knowledge base. Systems based on 
deep reasoning are slower but more flexible conqiared to 
shallow systems. 

25 Some researchers have even combined shallow and deep 
knowledge reasoning. It is hypodiesized by some that using 
"deep** representations of entities to be diagnosed is siq)erior 
to using en^iirical knowledge about assodations between 
malfunctioning parts of an entity and symptoms. The ratio- 

30 nale is that one caimot exhaustively catalog all such asso- 
ciations; and without such a catalog, a heuristic-based diag- 
nostic system becomes bdtde and fails when presented widi 
a case that it does not understand. On the other hand, deep 
knowledge representation is based on modds that are dif- 

35 ficult to construct— especially models that exhibit the tedi- 
nological intent of the designer. Ftohcr, it is unlikely that 
models will mirror the failure bdiavior of entities of any 
complexity, particulady with regard to jvoviding informa- 
tion about multiple perspectives. Moreover, models are 

40 likdy to be domain-spedfic, and only some fi'action of 
knowledge will be transferable from one diagnostic system's 
knowledge base to another. 

It is also recognized that rule-based systems become 
increasingly difficult to understand and Tnaintain as the 

45 number of roles grow. While a reasonable rule-based expert 
system shell can assist a domain expert in formulating 
cause-and-cffect rules, a collection of such rules typically 
will not function as an expert system, except in the most 
sinqile cases. To overcome these limitations, some expert 

50 system shells allow the encoding of strategic and object- 
levd knowledge as metarrules. This, however, requires 
extensive knowledge of the programming paradigm and die 
development environment Anodier dass of tools provides a 
search algorithm for a fiat problem-space rqnrcsentation. 

55 Although the problem rqnesentation is sin^lified, the 
seardi complexity for a problem of solution length L and 
search space branching factor of B has worst-case convex- 
ity 0(B^). Given die above arguments, what is the solution? 
Jt is proposed that in diagnosis it is actually more fruitful 

60 in cotain domains to concentrate on constructing models of 
belief organization rather than modds of physical entities. 
The concept of belief potentially has a wider scope than 
c^qilidtiy defined knowledge. Belief descriptions are usually 
generalizable and flexible in q>plication, whereas explidtiy 

65 defined knowledge has limited scope and is very brittie. 
In an q)proach in accordance with the present invention, 
the domain e^ert's knowledge about the behavior of the 
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environment and the general information processing tasks 
are represented as a Behavioral Hierarchy Knowledge Base 
(BHKB) (see FIG. 6). Next, a number of knowledge bases 
representing the expert's knowledge about the physical and 
structural hlcrardiy of the environment and the specific 
information processing tasks^ called Structural Hierarchy 
Knowledge Bases (SHKB), are linked to each terminal node 
of BHKB. Finally, the expert's knowledge about the asso- 
dation between symptoms, causes, functional characteris- 
tics of the environment, possible remedies, and the execu- 
tion of various specific tasks and procedures are rq)resented 
as brandies of the SHKB, implemented as Condition Action 
TVees (CATs), as shown in FIG. 7. 

Preferably, collective understanding about the diagnosis 
of a given fault is represented in a hierarchical fashion. 
Rqircsentation of knowledge as a hierarchy of networks, 
rather than by rules or models, has a three-fold advantage. 
Hrst of ail, system performance and incremental e?q)ansion 
is improved by eliminating the rule-base maintenance and 
rule interference problems. Second, such hierarchical 
decomposition provides access to intermediate states and 
thereby reduces the seardi space. Third, such knowledge 
representation is consistent with the expert's mental model 
of the environment and the reasoning process. Thus, overall 
development and maintenance effort is improved, as well as 
the predictability of the system behavior, independent of the 
amount of knowledge added into the system. 

A knowledge representation architecture in accordance 
with the present invention is composed of three levels, 
hierarchically ordered according to the principles of 
"increasing precision with decreasing intelligence," namely: 
the abstract behavioral level, performing genial informa- 
tion processing tasks; the structiral level, dealing with 
specific information processing tasks; and the action level, 
executing various specific tasks. 

A single knowledge base, called Behavioral Hierarchy 
Knowledge Base (BHKB), represents the high level abstrac- 
tion of the problem space. The BHKB is in fact a behavioral 
representation of the concepts and hypotheses associated 
with the problem ^ace. A number of knowledge bases 
representing ihc structural hierarchy of the probl^ domain 
knowledge are linked to each of the terminal nodes of the 
BHKB. These Structural Hierarchy Knowledge Bases 
(SHKB) represent more specific knowledge (hypotheses) 
about the environment and can be viewed as the structure of 
the problem domain and lower level problem resolution 
methodology. BHKB, on &e other hand, are generated by 
analogies drawn across domains and spplications, apply to 
a wider dass of problems, and dq)ict abstract problem 
solving strategies. Each SHKB represents a tool, process, 
process control, procedure, logistic, or service. Finally, diere 
is the action level, which performs various specific tasks. 
The diagnostic actions at this level are implemented as 
CATS. 

Note that diagnostic knowledge in accordance with the 
present invention is represented and processed as deep 
knowledge first (BHKB), then shallow knowledge (SHKB 
and CAT)- The set of CATs associated with each terminal 
node of SHKB represents explidt knowledge at the lowest 
abstraction levd. Such knowledge usually deals with the 
functional charact^stics of 1ht diagnostic process and the 
environment (tool characteristics, fault isolation, component 
testing, applying a specific remedy, etc.). TTie tri-level, 
hierardiical partitioning and representation of the diagnostic 
knowledge (BHKB, SHKB, or CAT)* shown in FIG. 7, 
enables a user to add or delete problems (SHKB), proce- 
dures (CAT), and/or specific instructions (nodes) easily. 
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without much effect on the system. In addition, it pemiits 
interactive diagnosis at three levels of abstraction. Thus, as 
an interactive diagnostic session progresses, abstract and 
vague dialogue is replaced with explidt and often exact 
5 interaction. 

This knowledge representation does not treat time-related 
situations (events) differently. Events are simply represented 
as time-stamped branches of their corresponding CATs, as 
shown in FIG. 8. The knowledge representation does not 

10 indude specific information on the acmal date the event 
might have happened, but rather how such an event should 
be taken into consideration in the diagnostic process. For 
example, the recent replacement of a faulty component with 
a good part should decrease the belief that the part is at fault 

15 This is a part of the knowledge base, but whether the part has 
actually been replaced is not a part of the knowledge base. 
Such information may reside in tool history logs or main- 
tenance databases ^^ch could be queried by the knowledge 
base. In some cases, the diagnostic system may ask the user 

20 to verify whether an action has taken place, and if the answer 
is yes, to identify and verify the date of the action. 

A (Condition Action Tree (CAT) is an "n-ary" directed, 
acyclic, gr^h with a root N^ and entry nodes N^, and L 
levels, each level having n^ nodes. Each node (except a leaf 

25 node) has n children nodes, as shown in FIG. 9. Here it is 
assumed that each node has the same number of diildren (n), 
but in reality (n) is the maximum number of children a node 
can have. A node may have more than one parent, but there 
is no notion of inheritance. For modeling simplidty, assume 

30 that a node has only one parent The presence of a link firom 
upwards node A to node B expresses the logical implication 
Vx. A(x)=>B(x). Three attributes are assodated with each 
node: node strength NS, node cost NC, and node rank NR. 
The strength NS^,- of node N^ is obtained from historical data 

35 and the opinions of expats, and represents the degree of 
belief or certainty that node N^ is responsible for the 
synq>tom(s) described at its parent node, ^t-i^(ceu(//n)y ^tj ^ 
the jth node in level i. Therefore, its parent is the (cen(j/n))th 
(ceUing of j/n) node in levd i-1. It is required that 

40 

M 

1r other words, the sum of the strength of all the children 

45 nodes of each parent node is unity. The node strength NSy 
represents a conibination of an expertrdefined endorsement 
and the historical information accumulated during use. The 
cost NC^ rqiresents the relative cost (in doUars, time, or 
other resources) of performing, testing, and verifying the 

50 conditions and actions described at the node. For computa- 
tional sm^jUdty assume O^NCf,^l for all i and j. NCy=0 
means no cost, time, or effort, while NC|/=1 means 
extremely costly, very time consuming, or very diflSculLThe 
NR is the ranking of a node among its siblings. 

55 Each sub-Usk identified in the knowledge acquisition step 
is mapped to a knowledge module, in^lemented as a G^. 
Each node in a CAT may have one or many possible 
branches associated with it Branches (diild nodes) are 
sorted according to their node rank NR, whidi reflects the 

60 most natural, useful, and meaningful ordering of all sibling 
nodes in a group when presenting them to a user (NR=1,2, 
. . . , 999). A node with NR=1 will appear at the top of an 
option list when presented to the user during the diagnosis. 
Each leaf node has two attributes: ^Tixed" and '*Not Fixed." 

65 Fixed is the confirmation by the user that the solution 
(procedure) presented by the CAT up to that point resolved 
^e problera Not Fixed, indicates that the solution did not 
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solve the problem or produce a favorable resolution. A node vices. Inference control is passed to &e appropriate module, 

in is either a question node or an action node. A At each step, any of the service object may be called upon 

question node asks the user for information or verification of to improve the quality of interaction, 

some attributes. An action node asks the user to pcifoiin Contiauing witti FIG. 10, from instruction 112 processing 

specific actions (measurements, component or attribute test- 5 dctcnnincs whether an exact matdi ctr good candidate exist 

ing cff verification, «c.) or calls iqjon services to perform ^ ground level space, 114 •'Exact Match Or Good 

some actions. In addition, each node contains the following Candidates?" If no match exists, then processing returns to 

information- *® previous abstractioo level to detomine whether all 

DQucstionitobcaskedortaskstobcpaformedtoconfirm I^KB °2;des havcbccn exhausted, 116 "SHKB 

or disconfirm a particular symptom. lO ^""'^"^I ^ ''''fT^it^^'^^''^ ^^1^' 

2) What (a clarification of queZns or actions). J^**^ structural nodes have been exhausted for the particular 
Tin. / . ~rr ^ . ^ ^ behavioral node then processing determines whether all 

3) Why (an explanation or justification for questions or ^^^avior nodes have been exhausted, 110 **BHKB 
.V , . . . . . , . Exhausted?**. Assuming that there is a partial match in the 

4) Additional informaUon to be presented (pictures, ground level space, then (he inference processor collects 
animation, video, audio). 15 120 "Coliect Mare Data.** If more data is 

5) Services (record session log, tools maintenance log, user available, then processing returns to instruction 112, while if 
log, etc) HQ additional data is available, processing determines 
A hierardiical knowledge processing algorithm in accor- whether the problem has been resolved, 118 *Troblcm 

dance with one aspect of the fffcsent invention employs a Resdvedr Alternatively, with identification of an exact 

hierarchy of abstract problem spaces, called abstraction 20 j^^jj^ ^t inquiry 114, processing passes to inquiry 118 

spaces, to focus a search process. Instead of attenq)ting to jf problem has not been resolved (ie., is 

solve a problem in the original problem space, called rejected), then processing returns to inquiry 116. Once 

"ground space," the present invention first searches for a resolved, inference processing is coii^>lcte, 122 "End." 

solution in tiie most abstract problem space to produce a Entropy is a commonly used measure for ranking infor- 

skeletal plan. This plan is then refined at successive levels in 25 j^^on squ^c^s. It is based on the assumption that the 

the hierarchy by inserting additional operators to produce a uncertainty regarding any variable x characterized by a 

complete sequence of ground-levd operators (see FIGS. ja-obahility distribution p(x) can be represented by tiie 

entropy fiinction: 

Given a problem de5crq)tion, the knowledge processing 

algoridun in accordance with the present invention proceeds 30 g^^^^ ^ _ j. p^^jLog^;^) 

as shown in FIG. 10. Hrst, the given problem, 100 "IVoblem ' 
& Domain Description," is mapped into tiie most abstract 

space, or "BHKB" (FIG. la), by deleting Uterals from the approaches fail to take into consideration the cost 

initial state and goals that are relevant to the abstraction of important activities associated with diagnosis such as 

space. Next, a solution is found ifit exists, 104 •'Exact Match 35 verification, availability, replacement time, tool down time, 

or Good Candidatesr, that solves the abstract problem, 106 ^tc In addition, cost is often not utilized in the first stage of 

'•SHKB.'* Each intermediate state in tiie abstract plan saves reasoning, but only in shallow knowledge reasoning, 

as a goal for tiie sub-problcnis at tiie next level in the Most entropy approaches by definition do not incaipoiate 

abstraction hierarchy. The algorithm tiicn solves each inter- any attribute (e.g., cost) except estimated probability. In 

mediate sub-problem using the final state of one sub- 40 order to accommodate otha attributes, tiie entropy function 

p-oblem as tiie initial state for fee next sub-problem, 108 ^^^3^ be modified. One modified form of entropy is called 

"Exact Matdi Or Good Candidates?". If "no," processing "usrful information," upon which the preferred entropic 

next determines whetiier tiie abstraction space has been measure is based. Given tiie node strcngtii NS« and node 

exhausted, 110 *'BHKB Exhausted?', ff again **no," pro- cost NCy, a node's entropy is defined in tarns of its 

cessing returns to instruction 102 where a next BHKB node 45 ^indren's node strengtiis and node costs as shown above, 

is obtained. If '"yes" tiicn from inquiry 110 processing Again, it is assumed tiiat a node has n children and tiie CAT 

updates the problem or the domain description. The process ^ levels, 
is repeated until the plan is refined to the ground level space 

(Goal, Evidence, Solution nodes in FIG. Ic), 112 "CAT." ^ 

Wth tiie domain knowledge encoded in tiie diagnostic 50 ffg^'f^^^Q^' j^^^y^^^Qf^iJO^^ 
knowledge modules, the diagnostic reasomng process fol- 
lows tiie maintenance, troubleshooting, or evaluation para- 

digm c^>ttircd in the knowledge engineering stage. A main- 0 g m: si 
tenance expert, for example, begins the diagnostic process 

by first observing and examining the overall performance 55 an] 
<^aracteristics of tiie equ^)ment. The proposed architecture 

can duplicate tiiis action by presenting a hierarchy of prob- Aa^w) = 1 

lem description menus from which the user selects. As the fc=((/-i)n>l 
user proceeds through these initial "problem identification** 

menus, tiie questions become more specific, and a descrip- 60 Whenever a user does not know the answer to a question 

tion of the problem is compiled. At each step this description (unknown), or cannot select an option from a presented list 

is matched against tiie entry node of all resident knowledge (conflict or incomplete information), the branch with tiie 

modules and ttie user is presented witii questions, data lowest entnipy Hy(NS, NQ is selected. This allows the user 

panels to be filled, multiple options from which to select, to continue with the diagnosis and be able to examine the 

option lists to rank, or tool parameters to be measured or 65 following steps. 

examined. The system then evaluates user responses and Given a problem description (malfunction, process 

requests and identifies relevant knowledge modules or ser- problem, product problem, etc.) and a problem domain (tool. 
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process, product, control system, task» etc.), the basic cycle organized, and converted to an on-line hypertext case-base, 

of the algorithm is to find (interactively and cooperatively) Such a case-base could serve as a fast path for expert users 

the problem domain (BHKB), problem category (SHKB), who know what they arc looking for and do not wish to go 

problem type (CXT), and a resolution (CAT leaf node), as through a step-by-stq) diagnostic process. Bach case is 

described above in connection with FIG. 10 and outlined s preferably organized iato five fields; namely, ca5e_Jitle, 

below. symptoms, actions, problem, and solution (described 

Step 0: Given BHKB, SHKBs, and CATs, initialize all and below). Users can query the case-base on any combination 

evaluate all entropies. of keywords in any field. The case-base preferably com- 

Stq> 1: Present descrq)tion of design environment, problem, prises a hypertext document and provides fuzzy search 

and symptoms to BHKB. lO capability based on the word root as well as wild diaractcr 

Step 2: Starting at the root, select a node in the next level and mult^le word seardiing. In case of multiple matches, it 

which satisfies the design environment attributes opens up to the first match, with the case_jtitle of all 

(corresponding to the level) given in Step 1. If there is no matched cases displayed in a selectable list box. 

exact match, select the node with lowest entropy. Update CASE_TrrLE: Problem class, group and category 

entropy list by deleting tiie selected node from list This 15 SYMPTOMS: list of all symptoms reported by the user 

node is now the root of the remaining tree. If the entropy during the diagnosis 

list is exhausted, return to Stq> 0. ACTIONS: List of all actions recommended or performed 

Stq> 3: Repeat Step 2 until a terminal node of the BHKB is by the user 

reached. If there arc multiple terminal nodes or PROBLEM: Final diagnosis of the real cause of the symp- 

candidates, rank them based on the entropy of the root 20 toms 

node of the corresponding SHKB. Then activate a can- SOLUTION: The corrective actions which fixed the prob- 

didate SHKB. lem 

Step 4: Starting at the root of the active SHKB, select the By way of example, FIG. 11 depicts a computer system 

node in the next level which matches the problem and upon which the expert system (and/or multimedia/ 

symptoms given in Step 1. If there is no exact match, 25 hypermedia system) in accordance with the present inven- 

select the node with lowest entropy (update entropy list). tion can reside. This computer system may comprise an IBM 

If the entropy list is exhausted return to Step 2. PS/2 microconqmta- system employing a 386 or faster 

Step 5: Repeat Step 4 until a terminal node is reached. If processing chip. In addition to a hard disk and floppy disk 

there are multiple terminal node candidates, rank them drive, the computer system has a video adapter, a video 

based on the entropy of the root node of the corresponding 30 display monitc^, a keyboard and a mouse-type data manipu- 

CATs. Then activate the candidate CAT with the lowest lation device, all for user interface with tiie expert system. If 

root node entropy. desired, the expert system (and/or multimedia/hypermedia 

Step 6: If the problem is resolved, processing is done. Else system) could be implemented on a portable system, such as 

if the solution is rejected by die user or there Is an a laptop or notebook conq)uter. 

unknown problem, and the CAT list is not exhausted, 35 Architecture frameworks for an expert system 150 and a 
present the next CAT. If the CAT list is exhausted, update multimedia/hypermedia system 152 in accordance with the 
NS^ of the terminal node of SHKB corresponding to the present invention are depicted in FIG. 12. This object- 
rejected CAT. Reevaluate entropy for all parent nodes of oriented modular architecture is comprised of a collection of 
this node, until the root of the SHKB is updated. Then int^acting knowledge modules and *^itilities," inchiding 
update the node strength of the corresponding terminal 40 training modules, on-line documentation and service 
node in BHKB and reevaluate entropy for all parent nodes modoles, extonal applications, and databases arranged 
of this terminal node, until the root of the BHKB is around an inference kernel. As used hearein, a *^itility" is a 
updated. Return to step 2. module containing non-inferentially accessible inforiDation. 
When a CAT at a terminal node in SHKB does not As described further below, training modules are multi- 
produce a satisfactory solution, or the SHKB linked to a 45 media applications which provide on-line training based on 
terminal node of a BHKB does not converge, the terminal user experience level (beginner, novice, intermediate, 
node strength needs to be updated to reflect this scenario. expert). The user profile database records user expertise 
Updating a terminal node strength affects all its siblings' levels, tasks and procedures that users are certified for, 
node strengths. The heuristic described below reduces the certification dates, and work experience histories. The user 
node strength as a function of the weakest node, but ke^s so profile database is maintained by an area administrator and 
the sum of all node strengths unity. Once node strengths are is accessed and updated by the certification modules, 
updated, new entropies are evaluated and propagated On-line documentation provides information on processes, 
upward to the root node. tools, logistics, and other aspects of the task. User guides. 
Step 0: Update NS^ of the node N^. operational manuals, manufacturing procedures, dcrigo 
Step 1: Identify the total number of siblings (q) If N^ is the 55 guidelines, etc., are implemented as hypermedia documents. 

only child, return 0. Service modules provide various communication and pre- 

Step 2: Identify the node with minimum node strength sentation services between the user and the system. These 

QiSfj^. services include displaying high resolution images, playing 

Step 3: Subtract Z=}^SuJq from Ns,;^ {NSfy=NS^Z). audio segments, playing motion video and animation, log- 

Stq> 4: Inaease the node strength of all siblings of by 60 ging tool, process, and area histories, establishing session 

Z/(q-l). defaults, logging user feedback, and various report printing 

In some cases, users may not want to go tiuroug^ a facilities. Diagnostic knowledge base modules provide the 

stcp-by-step diagnostic procedure, since the user may know interactive multimedia tool and process diagnosis described 

wliathe is looking for, e.g., the problemis known, but all the above. A user accesses the inference kernel through a 

steps involved aren't To address this situation, summaries 65 graphical user interface. 

and conclusions are preferably generated at the end of Ckaphical user interfaces (GUI), nmltiple screen displays, 

diagnostic sessions. I^ addition, cases can be collected, and windowing capabilities have provided powerful possi- 
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bilidcs far user interface design in recent years. Many 3. A dcsigaor's tnodcL 

researchers in the field of human-computer interaction and A user's conceptual model is a mental model A program- 
cognitive science have extensively studied the effects and mer's model and a designer's model are explicit and con- 
requirements of such an interface environment on users as sdously designed. Each model rqnresents a differmt audi- 
wdl as system design. 'Wi&i the advent of windowing and 5 ence's perspective of user interface. FIG. 14 shows a 
multiple screen displaying capability, additional issues and summary of CUA and the various interface modds it sap- 
concerns have to be taken into consideration in design of pofts. 

computer user interfaces. In addition to a consistent dialogue and interaction, the 

The user interface is a graphical, window-based interface window structure proposed here also provides a consistent 

consistingof visual representations and interaction facilities lo windowing structure. As shown in FIG. 15, the screen is 

(e.g., see FIG. 13). Interaction with the system is through a divided into four sections. The system window (upper Idl) 

menu-driven, point and cJick interface. Manual data entry contains the system dialogue, action bar menus, options, 

and typing are kqit to a minimum. The user interface textualinformation to be displayed to user, and the standard 

methodology is based on IBM's Conunon User Access IBM I^esentation Manager service icons (minimize, 

(CUA) architecture. CUA is a set cf guidelines to be used to 15 maximize, scroll up, scroll down, and system menu). The 

design user interfaces that are easy to learn and use. The summary window (lower left) appears at the end of a 

CUA ardiitecture delines the worlqilace environment for diagnostic session or when a conclusion is reached, 

programmable wcH^kstations. Hicse guidelines are based on Warnings, notices, and inq^ortant messages are displayed in 

user-interface and object-oriented relationships and have the warning window (upper right) in a bright color to get 

been supported by human factors research and usability 20 attention.Allgraphics, digitized images, pictures, diagrams, 

testing. They specify common user-interface components slide shows, animations, and scanned schematics ^pear in 

and techniques, as well as guidelines for applying them. The the gr^hics window (lower right). If there is more than one 

pull-down menus and system menu are consistent with IBM picture to be displayed, multq>le graphics windows are 

OS/2 Presentation Manager standards and the IBM CUA opened on top of eadi other with a 20 pixel offset toward the 

guidelines, as shown in HG. 13. 75 cento* of the screen. Any graphics window can be brought 

The appearance of and interaction with the interface are up to the front by just clicking on any exposed part of it AU 

consistent throug;hout system applications. Menu options windows are opened and closed by the systm during the 

and choices are selected by double clicking. Data entry is session, but the user can also minimize, maximize, or dose 

performed by typing and clicking on the **0K" button or any of the windows except the system window at any time, 

pressing the ''Enter" key. Multq)le choice sdections arc 30 The exception to the above is the external application 

made by placing ttie cursco* on the selection and double windows wtdch open in the center of the screen, but can be 

clicking on the mouse button or by pressing the ''Dib** key moved, minimized, or maximized, 

to hi^ilight the selection and then pressing the ''Enter" key Since the systems proposed herein might be used in a 

to make the selection. All user interface functions are semiconductcB' manufacturing environment, an important 

accessible via the keytx)ard as well as the mouse (a CUA 3S issue in the design of the user interface is color consider- 

requirement). ations. For example, semiconductor manufacturing includes 

There has been a lot of research in the field of user areas with soft yellow lighting (dean rooms), soft white 

interface design prindples, styles, and standards. Cognitive lightmg, and bri^t white lighting, '^^dow backgrounds, 

and computer sdcntists have produced valuable and practi- higjili^tings, text, and image annotations have to be visible 

cal user interface design principles and guidelines. Several 40 and distinguishable in all these environments. Although a 

sets of standards and guidelines have also emerged in the single set of colors suitable far all occasions and all envi- 

computer industry, such as IBM*s Conunon User Access ronments has not been found, some color combinations are 

(CUA) guidelines and user interface design principles. more smtablc for some areas. For cxaniq>le, multimedia 

Common User Access (CUA) describes principles, documents with considerable textual information are much 

components, and techniques for user interface design in 45 easier to read and focus on in dean room environments 

general, as applied to a variety of software products for a ^en the background is black, text white, and highlights are 

variety of operating environments. It also describes ^ bold large font In addition, human factors analysis has 

process of designing a product with a CUA interface in revealed some problems and limitations in use of colors 

particular. The CUA user interface is a graphical interface whidi have to be taken into consideration. For example, 

that incorporates dements of object-orientation, i.e., an so nmy color-blind people have trouble seeing red and green, 

orientation in whidi a user* s focus is on objects and in which Some see no color at all; others caimot distinguish blue from 

the concq)t of q>plications is hidden. The CUA user inter- purple. For these reasons, use of color for highlighting is not 

face is based on pindples of user-interface design, on recommended and should be avoided, 

object-oriented rdationshq>s, and on fidd experience and In designing the User Interface (UI) several other factors 

user testing. 55 have to be taken into consideration. Most computer-based 

The CUA guidelines provide information about how the applications and services with which operators deal are 
model on whidi a product is based should be conveyed to a either on charactex-based non-programmable terminals, with 
user. The CUA guideliites also indude specific details about display of one window at a time, very littie mouse use, and 
designing and devdoptng conqiutcr software and user inter- few selectable options, or are designed as character-based, 
faces. Products that adhere to the CUA guidelines and 60 single window (full screen) q)plications. So, the operators 
undergo usability testing should be coiiq)aratively easy to using the system should be empowered but not over- 
learn and use. As a result, users of products with a CUAuser helmed by too many choices, options, and windows, 
interface should find the products easy to use, efficient, and Therefore some features of the UI, such as window position 
appropriate for the tasks tiiat users want to accomplish. With and size, can be fixed and predefined to reduce the required 
respect to the user interface, three models come into play: 65 user intervention. 

1. A usa*s conceptual model; Throughout the UI and application development, 

2. A programmer's model; and terminologies, keywords, and buzzwords known to the 
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manufactunng penonnel should be used. Such tenninology hand, refers to the backward chaining of the same event All 

can be compiled during meetings, intaviews and/or supcx- arc strengths (S^^ S^c» ^ad^ • • • ) start at 1, and are 

vised interaction sessions. Tenninologies, symbols, and inaeased by one eadi time the nodes are selected. In essence 

acronyms used in vendor manuals and troubleshooting this rule mate the assumption that the more a link has been 

guides can also be utilized to assure consistency with 35 , . _ , /TT , j • 

Lrentmodeofopcrationandtoprovideauser&iendlyand used, the more likdy it is to be useful now and in the future, 

familiar work environment Onct node A is activated, the next path (AB, AC, or AD, . 

Documents incorporated in the diagnostic modules, as • . ) will be affected by the strength of all possible arcs (S^, 

well as on-Hne documentation and training documents Sax>* • • • )• strengths are then normalized and 

should provide 'linguistic-based" search and match fiinc- converted to node ranks. 

tions. This wiU aUow seardies for words based on word root Coordinating text, graphics, and images in a multimedia 

and »Svfld cards". This aUows sear<± and inatch for infor- environment is very important to effective presentation and 

mation rather than just an exact match. This hnguistic-based ._r*t. r™. iSL j- • 

search and match concept exists in IBM^s B^k Managa managementof the subject matter. The mulb^ 
p-oduct. In addition, a table of 'linguistic hedges" can be ^ methodology of the present mvention is based on a pre- 

devdoped which contains lists of equivalent and similar defined presentation scenario and is implemented in one of 

phrases and keywords. This table can be customized for the predefined templates. 

manufacturing in general, or semiconductor manufacturing Specifically, multimedia modules are partitioned into 

in particular. "logical pages." A logical page (e.g., HGS. 16a-l^ & 17) 
A user interface is considered adaptive if it changes in ^ ^^^^^ ^ of images (still, video, animation, graphics) 

somewayinresponsctoti^personwto ^ associated description (text and audio). In other 

Sudi an interface uses information about the user's goals, . , • , j * j n Jl* ^ » 

the current active objects, and the strength of association words a l^cal page is the predefined coUedion of aU 

betweenthese objects to dynamically order the list of panels, related information (text, graphics, images, audio 
items in a list box, or topics presented in a imiltimedia or ^5 ^^^s^ction, animation, video dips) which can be made 

hypermedia document (discussed below). During available to the user. The appearance, format, and access to 

consultation, the user is presented with a list of relevant and logical pages is preferably consistent throughout the system 

relative options at each step. At the beginning of the session, Logical pages are related to each other via hypertext and 

these lists are very high level and not very long. As the hypetgraphic links. 

session progresses and travel occurs further down the das- 3Q jo assure a con^lete and accurate display of a logical 

sification tree, the number of options could be quite large. page (regardless of how it is addressed), it can only be linked 

In some cases it becomes necessary to sort the lists to via its primary window. A logical page, similar to a page 

presented to the user, based on likelihood of usefuhiess or ^ ^^^^ contain many different objects of interest, but 

relevancy, and based on past experiences. Note that in canonly be opened by a caU to its primary window. All othff 

navigating tiirough the CAT the us^ might ^^^ve to select 35 ^^^^ ^ ^ ^ ^^^^^^^ automaticaUy 

one of several opuons at a node. With an unordered hs^ the prin^ window or he pop-up windows whidi 

user has to seardi and try on average options (N=tota^ activated via hypertext or hyper^pMc links. When a 

number of options to select ficom). With an ordered hst it . , . ''^^ ... Z*™! f i- 1 l i.- 

would take 2M/N (i.e., M/(N/2)=2M/N) as much option pmnarywmdow is opened (via hp<^cxthnk, hy^^^ 

scrolling before the user reaches the target option (ie., the '^^^^^"^ ^^'^^^^ ^f^""^^ ^ 

correct one). M is tiic number of options above tiic target result of a seardi) aU its dq)endent secondary wmdows are 

option in the ordered list Of course, it is assumed that opened automatically. The following is the source code for 

M<N/2, and in fact results reported by other researchers in tiie logical page shown in FIG. 17. 
the field of adaptive tutoring systems and intelligent inter- 
faces support this assumption. 45 , 

In the present user interface methodology, the items in an '^^'ctA^^nc toc=i dU='Lo^_j)g' 

option list box are ordered according to the predefined node objectmme=Xo£ical_Page— Main* ob)ectid?'&lex>Ft'. 

rank (NR) and node strength (NS). Node rank spedfies the Wafer Polisher 

order in which options are displayed in the list box. This res=io i=kft y=bo^j«dtfa=409& l«igl^iOO% s^xip=i 

initial ordenng and node Strength assignment IS based on a 53 riixjk jcflype=hd its=il group=2 

study of existing cases and feedback from experts. vpx?=Tight vpy=top vpci6og& vpcy^ 

Preferably, the system should be able to dynamically auto dependeat 

enhance and reevaluate the node ordering, node strength and '^^^^ '^tto^Uo* -50% 

eventually knowledge base ordering. In otiicr words, a form ]^d^o^t ^'^^ 

of unsupervised learning is desired based on existing :p.:hp2lk)lishAxxu:el9l 

resolved cases. In addition, the network should be able to :pJ»olish Arm rotates Canicr to (primaiy) :link.reftypcp4id 

track its utilization and adjust and reevaluate its node !f^^* *i m . r 1, n^r ». a 1- • ^ 

. . . , . .. 1 ^ i> PoTasmng Station Flate:elu]k.Poliui Ann lowers earner to 

ordermg and node strength accordmg to the frequency of ^ (primaiy) polishing begins. 

use. Assume arc AB is a section of a knowledge tree. :pJiiik iefid=wa£er ieftype=&.m£erelink. canier rotatkm 



60 



speed is programxDable . . . 



I Sab i 

.jH. U :h2 le^U ctrlaiea=iioae hide noseaich ooprict tiUebai=Qaiie 

^ ' ndcs=nGiic. 

;yrtWOlfc Dames *&2ZILb^^)* Itnltfil^ ^'flTm bgl'. 

S^ means tiie degree of association between node A and B '^^^"''^"'"^^ ^ '^"^ titw,ar=«>oe 

throu^ arc AB. This is the link strength, or the degree of 65 :artwoifc namc=*currenLbn5*. 

evidence or certainty that following arc AB will advance :fii i*=vafer. 
problem resolution toward a final solution. S^aj on the other 
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-continued 

ItuD sEoe cut fiom a cylinder of a fimgle-aystal silican, vgan 
which . . . 

leuserdcx. 



A logical page always has a tat winctow which serves as its 
piisiaiy window, except full screen non-textual pages 
(image, graphics, audio, motion video, animation, external 
applications) in which case the page title is considered the 
textual window. A window can be assigned vertical and 
horizontal scrolls, as well as title, background and fore- 
ground colors, font style and size. It is reconmiended that 
text windows be assigned only vertical scrolls and allow the 
Information n:esentation Facility (IFF) conq>iler (marketed 
by IBM) to fit and wrap the text in the window. 

In addition to hypertext links, interactive multi-function 
hypcrgraphic linVx arc also preferably provided. Images or 
con^onents in an image can be linked to other sections of 
the document, other images, or even other documents via 
hypergraphic links. '*Hot-^ts" defined for an object or for 
an entire image are used to make the object or image 
selectable. Eadi hot-spot has a function pop-up menu which 
is selected by clicking on the object Hien a specific function 
is selected by cUcking on the desired function button/ 
symbol. 

To accommodate various situations and to maintain con- 
sistency among multimedia modules, a set of "Logical Page 
Templates" are defined (e,g., see the partial lists of FIGS. 
X6a-16p & 17). These templates provide consistent facilities 
for embedding various multimedia and hypermedia utilities 
and functions in applications. These logical page tenq)lates 
facilitate sharing of multimedia modules among applications 
and also among other similar systems developed in other 
parts of a company. 

A complete and integrated methodology and architecture 
for development of intelligent interactive multimedia/ 
hypermedia computer-aided training and certification appli- 
cations in accordance with the present invention is next 
described (see FIG. 20). The metijodology takes full advan- 
tage of modem multimedia workstations. Hypertext, 
hypergraphics, high resolution images, audio, motion video, 
and animation are technologies that in synergy are far more 
useful than each by itself. The modular and object-oriented 
architecture provides an interactive and coopo'ative 
computer-aided training and qualification development 
environment. The Control and Management Knowledge 
Bases (OlKBs) serve as an intelligent tutoring system 
framework. A major departure of the £9>proadi described 
herein from previously reported computer-aided training and 
intelligent tutoring systems is the seamless integration <^ 
training, certification, and on-line documentation with 
knowledge-based control and management 

Existing multimedia training methodologies usually 
require a large initial course prqiaration investment and are 
suitable only for relatively stable subject matters. Such 
applications usually require specialized, and often 
expensive, hardware and software. The interactive multime- 
dia training and certification methodology presented herdn 
is based on standard IBM OS/2 features and a set of novel 
tools and tedmiques (described above and below), and 
provides modular, easy to upgrade, customizable training 
modules without any additional dedicated hardware or soft- 
ware. Applications developed under this methodology and 
ardiitecture reside entirely on hard disk ca* removable/ 
rewritable storage medium (i.e., optical storage) and can be 
used in stand-alone or network configuration. 
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Traditional Computer Based Training (C!BT) and 
Computer-Assisted Instruction (CAI) teach a subject by 
offering corrective feedback baseid on a large, pre-stored set 
of problems, solutions and associated remedial advice. The 
5 most noticeable problem with CBT and CAI is the amount 
of time it takes to produce effective courseware as all 
possibilities need to be antic^ated in advance by the authoL 
Some Intelligent Tutoring Systems (ITS) have addressed 
this problem by providing an environment where problems 
are generated automaticalty. Such "generative" systems 
need only be given general teaching strategies and they 
could produce a large number of interactions without all of 
them being explicitly programmed in advance. However, 
most of these systems lack a dear rqiresentation of the 
knowledge tiiat tiiey are teadiing. 

In such systems there is a mismatch between the pro- 
gram's internal process and tiie trainee's cognitive pro- 
cesses. The orainee using CBT and ITS is usually denied any 
chance of using his/her initiative in guiding the learning 
process. The CBT and CAI strategy is 'learning by being 
^ told". However, researdi has shown that 'learning by dis- 
covery" and 'learning by doing" are in many cases more 
effective strategies. A main goal in the training methodology 
described here is to reduce intimidation and to put the trainee 
in control of learning. Instead of sitting in a classroom 
^ listening to an instructor, or following an operator on the 
manufacturing floor (who is usually fighting fires and the 
last thing on his/her mind is training), the ti^ees work 
individually on computers that combine text, high resolution 
color images, grq)hics, audio, animation, and motion videa 
This enviromncnt is characterized by student sequencing 
through course matfrial in a self-paced manner, while being 
monitored by the systeoL The goals in developing the 
proposed training methodology can be summarized as: 

Shorten tiie training process. 

JnxpiQvc the training quality. 

Provide consistent training across the board. 

Produce modular and portable training courseware. 

Effectively deliver and monitor tiie training objectives. 
40 Enable domain experts to easily develop and modify 
multimedia training courseware by providing them 
with architecture, m^odology, and necessary tools 
and techniques. 

As noted initially, manu^cturing lines can be serviced by 
45 hundreds of operators, production technicians, and mainte- 
nance technicians. C^ently, a multitude of methods are 
used for educating and assisting manufacturing personnel 
These methods include informal, unstructured training 
sessions, printed **in-house** manuals, sit-down classroom 
so courses, and walk-tlmi orientations, to name a few. In such 
an environment, the student population grows widi every 
group of new employees (ten^xsrary or permanent). This 
student group is large and decentralized. They caimot all be 
released from the manufacturing line simultaneously to take 
55 a standard training course. 

A majority of the process engineers, process technicians, 
and senior operators interviewed en^asize that their major 
concern is &e heterogeneous work environment within 
vMdi they worL In addition, it is emphasized tiiat the 
60 current training and certification methodologies are often 
inefficient and time consuming due to usual operator tum- 
around and frequent dianges in the process and tools 
en^loyed. In most cases, the new operators are trained by 
current operators and process technicians who have their 
65 own styles and biases. In addition, most of die information 
requirol for their regular activities arc textual documents 
(softcopy or hardcopy). 
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latcractivc nmldincdia training and ccrtificatioa offers modules arc usually used for Beginno- and Novice training, 
consisteiit presentation of the subject matter, on a flexible 24 where the trainee needs step by step guidance, momtoring 
hour per day, seven day a week schedule. The subject matter and feedback. They are also used in dangerous situations or 
delivered to the trainee is guaranteed to be consistent when when dealing with hazardous matmal, 
such a methodology is used. Thus, a "Monday Morning s Passive training modules can also provide step by step 
Syndrome" is avoided, i.e., y^exc some of the subject matter instruction and procedures, but they neither enforce it nor 
presented by a line technician to new hires might be a little monitor the progress. Such modules are suitable for Inter- 
sketchy or missing altogether. Using such an on-line mediate and Expert users who may not require step by step 
conq>uter-ba5ed training methodology also eliminates the control and guidance and do not have to be monitored as 
back-level problem. Currently, using printed documents one lO closely as Beginner and Novice users. Whenever multime- 
is never sure if a line operator is using the most current dia and hypermedia modules are accessed directly, they 
revision or whether an operator's training had covered the provide passive training, whereas if these are presented via 
latest version of process or tool upgrades. By making the diagnostic knowledge bases or the CMKB, then active 
information available "on-line," and integrating training, training is performed. 

certification and documentation, better control is obtained 15 Con^uter-based training and intelligent tutoring systems 

over what information is being used and what needs to be are usually based on a single and simple student model, 

disseminated. Regardless of the familiarity or lack of familiarity of the 

The architecture presented below provides an intelligent, trainees with the topic, all go through the same training 

interactive and cooperative con^uter-aidcd training and process. A new hire who has never worked in a similar 

qualification framework and can be classified as an Zntelli- 20 environment is given the same training material as the one 

gent Computer-Aided lYaining (ICAT) or Intelligent who has many years of related experience and might have 

Ccmputer-Aided Learning (ICAL) architecture. The object- been transferred to this new assignment from a similar area, 

oriented modular architecture is comprised of a collection of Catification procedures are usually conducted orally in an 

interacting and cooperating modules arranged around the ad-hoc fashion and are subject to a trainer's judgment and 

Control and Management Knowledge Bases (CMKBs), as 25 biases. In addition, there is no formal methodology for 

shown in FIG. 20. increasing the responsibility of a trainee as the training 

T^^ining modules provide interactive multimedia training proceeds. In order for a coii9)uter-aided training system to 

fortool,proccss, or business practices, as well as orientation acquire the necessary flexibility, it should distinguish 

for different areas or lines of business. Documentation between several types of students and structure the training 

modules provide on-line hypomedia documents covexing 30 material according to the students* needs and background, 

all necessary information on tool, process, process To address this problem and to provide a mefliodology for 

parameters, logistics and various other application and data- presenting and managing the training material to students of 

bases which the end-user (operator, technician) comes in varying background and expertise, a new multilevel student 

contact with. Certification modules provide an interactive model has been developed (see FIG. 21). Hiis new model 

knowledge based test, evaluation and feedback environ- 35 advances a trainee's rank as the training progresses and as 

m&nt CMKBs provide coordination, access control, and the trainee inouproves his/her certification test score, 

user management and serve as an intelligent framework for Expert technicians and operators were interviewed to 

the training and certification process. CMKBs act as a identify efficient methodologies for training the manufac- 

liaison between the trainee and the multimedia information turing personnel in their area. In other words, how should the 

modules and qualification procedures. In fact, CMKBs 40 training material be presented and managed for students of 

depict a supervisor's role, and provide control and monitor varying background and expertise? Four levels were iden- 

access to information and generate study plans according to tified: Beginner, Novice, Intramediate and Expert (see FIG. 

a student's background and progress. 21). A trainee rated as a Beginner, requires the most assis- 

A user profile database includes various information on tance and step by step guidance, whereas an Expert only 

the manufacturing personnel in each area. For example, it 45 needs to be able to locate information rapidly and efficiently, 

might include employee number, name, department, tools A student/trainee is assigned an initial level by the area 

and processes for which the employee is certified, date supervisor. The student advances to the next level 

certified, level certified (Beginner, Novice, Intermediate, (increasing responsibility and access to information) as 

Expert), and any expertise and authority level (system he/she passes the certification procedure for that level. As 

adnunistrator, production technician, maintenance so the user advances, the detailed procedural information is 

tectmician, operator, . . . ). Any module can query the user reduced, but the conceptual and deep knowledge 

profile database directly, but the updates and modifications (declarative and procedural knowledge) of the process is 

are managed by CMKBs or directly via Gr^hical User inaeased. A Novice user knows the steps involved in the 

Interface (GUI) by authorized personnel (e.g., system operation, an Intermediate student should know the cardl- 

administratoi). 55 nality and temporal relationship to the st^s. An Expert user 

Active training modules provide interactive step by step would also understand the taxonomy of the tasks and 

guidance and monitor user responses and adjust/react procedures. 

accordingly. These modules usually cover the more complex Sudi a multilevel student model reduces the danger of 

scenarios where the system s^es as an active participant in trainees getting involved in potentially dangerous or destruc- 

the process, analyzes user responses and formulates a oor- 60 tive tasks which they may not have adequate training and 

responding action based on tiie response and the ovciall preparation for. In the existing training methodologies, once 

goal. Active training modules could ask trainees to perform a trainee is certified die trainee could be assigned to process 

specific tasks, examine tool con^nents, review process products although he/^e may not have had any training in 

procedures or make specific tool/prooess/product measure- process or tool troubleshooting. In most cases, ignoring 

ments before continuing to the next step. These modules are 65 initial signs of problems or not being able to respond to 

structured as implication networks or decision trees, and arc problems quickly, could be very costly and result in tool or 

implemented as Condition Action Ttees. Active training process shut down. On the other hand, in some situations, 
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initiatives by operators who have not been adequately section nsing a link tag (dink reftype=^dies=100.click here 

trained and infonned of dieir le^onsibilities might result in to review Polish Arm sectiomeiiiiL). Once the docoment is 

scraps, reworks, damaged tools and products, or an out of conq»iIed, (he text (**Ciick here to review Polish Arm 

spec process. In the multilevel methodology presented section**) is uniquely highlighted so the user knows it is 

herein, CMKB modules keep track of the user level and last s selectable. Hcturcs can also be embedded into the document 

certification date, along with scheduled training sessions and as artwoik (:artwoik nam&=*arm.bmp* linkfile=*armJigl*.)- 

topics. In addition, these modules recommraid tasks suitable The Mnkfile contains the descrq)tion and coardinates of 

for each operator and decide whe&cr die operator should hot-spots defined for the image. Eadi hot-spot can have up 

have access to process and/or tool diagnostics. to 4 functions associated with it, which can be selected from 

To summarize, the certification process is interactive and lO the hot-spot function pop-up menu, as shown in FIG. 22. The 

cooperative and is geared towards process and tool operator hot-spot functions menu is selected by diddng on flie object 

certification in general, and manufacturing operator certifi- of interest The hot-spot functions are in fact secondary 

cation in particular. The certification procedures test a train- hot-spots within the primary hot-spot window. These func- 

ee*s progress, track perfcHinance and help the trainee con- tions are links to other parts of the document (textual 

centrate on problem areas. The feedback and tracking is description, dose-np picture) or to external applications to 

capabilities are responsive to the concerns with existing show an admation, display a video dip, or play audio (e.g. 

certification and training procedures, which lack a meaning- :link reftype=launch objec^'audio.cxe* data=*calibrtaud\ 

ful and unintimidating feedback mechanism. In addition, to Click here to listen to calibration procedures. :elink.). 

reduce intimidation, enhance learning and increase trainee One processing embodiment for creating a hypcigr^hic 

involvement, certification is preferably expanded beyond a 20 function pop-up menu is presented in FIG. 23. The process 

simple pass or f aiL Certification modules can be considered begins, 200 "Start," with a "hot spot* • being defined for the 

an extension of training modules by providing meaningful image, 202 '^entify Hot Spot For Image.*' Next, a default 

feedback, suggesting topics to be reviewed and providing hypergraphic function, such as text, dose-up, video, etc., is 

general conmicnts, as wcU as constructive criticism. preferably assigned to the hot spot, 204 "Assign A Default 

Preferably, a new user needs a password and a valid user 25 Hypergraphic FCT. To Hot Spot" 

Id to take a certification test. New users are assigned Thereafter, four hot spots are defined within a main hot 

USEELJLBVEL="Beginnef*, unless a different level is spot pop-up menu, using essentially the same procedure as 

assigned to them by the area supervisor. Once a trainee has that employed to define the main hot spot (disoissed below), 

reviewed all the required materials and procedures for the Each of the four hot spots serves as a menu button, and may 

area, tool group and level, the trainee can be given an on-line 30 comprise a square, for example, witii a side length of 

certification test. The test might be comprised of about approximatdy twenty percent the main hot spot hdgjit 

20-50 questions (multiple choice and True/False) which length. In the embodiment shown herein, the four hot spot 

may be answered in any order. The trainee can review an buttons are stacked on the right-hand side of a main hot spot, 

up-to-the-point status summary which shows the user 206 'Define Four Small Hot Spots ^tfain the Main Hot 

responses to questions and whether a question has not been 35 Spot** Next, hypergr^hic functions are assigned to the 

visited yet (Not Answered) or no answer has been selected small hot spots, such as text, dose-up image, audio, video/ 

for it yet (sk^). Once the test is turned in, the CMKB animation/slide show, 208 "Assign Hypergraphic FCTs. To 

evaluates the responses and generates a report. For each Hot Spots.** Inferably, hypergraphic functions are assigned 

question answered incorrectly, the system lists topics, in a fixed order. Hnally, if one of the functions is not 

documents, or training modules to be reviewed. 40 available, then the corresponding link is assigned a pop-up 

Tool operating and maintenance manuals, process footnote with a message to that effect, 210 'Tf FCT. Not 

documents, computer system user guides, safety and emer- Available Provide Footnote Message.'* This con^lctes pop- 

gency procedures, and hazardous material handling and up menu setup, 212 **End." 

disposal procedures are preferably convoted to hypertext A process for creating a hypergraphic hot spot link is next 

documents and made available as part of the on-line docu- 45 presented with reference to FIG. 24. Inquiry is initially 

mentation. These documents are accessible from training made, 220 "Start,** to ascertain whether the image is an OS/2 

and certification modules, as well as directly firom the bit map or metafile, 222 'Image=6it Map Or Metafile?'* If 

system £ront-end. Ext^nal applications (e.g., diagnostic '^no," then a conmiercially available conversion program 

expert systems) can also utilize these documents. A docu- such as IBM*s OS/2 Image Support program is employed to 

ment can be opened up to a specific topic or section. All 50 convert the image to a bit map or metafile, 224 "Convert 

documents may also be searched for a keyword or topic prior Image To Bit Map or Metafile.** Once in proper format, the 

to opening. In case of multiple matches, it opens iq) to the image is loaded into any commercially available image 

fn:st match, with ttit section title of all matched cases editor, 226 *lx>adlniage Into Image Editor.** Next, the pixd 

displayed in a sdectable list box. A hypertext maintenance (x,y) coordinates of the lower left comer of a rectangular 

log can also be set up, so all activities related to tiie tool, 55 area to be defined as die hot spot are identified, along with 

process, or business practice can be logged in for shift-to- the width and height of the area desired (cx,cy), 228 **Iden- 

shift communication and long term documentation. In tify Kxels (X,Y)+Width And Hci^t Of Area (CX,CY) To 

addition, periodic and long texm maintenance activities, tool Comprise Hot Spot** 

shut-downs, as well as technician/operator vacation plans Hie type of action to be Unked to a particular hot spot is 

can be scheduled and posted on this log. 60 then selected. For example, jump to another section of a 

Multimedia documents and training modules can be document->R£FTYPE=hd, open a pop-up footnote 

designed and devdoped using C, Presentation Manager window->R£FrYPB=fn, launch an external application 

(PM), Dialogue Manage (DM), OS/2 Information Rresen- A->REFrYPE=launch, 230 "Identify Action When Hot 

ration Facility (IFF) tagging language, and IFF conq>iler SpotSdected.'*If desired a s^)arate link file may be created, 

(IFFC). Each section in the document is assigned a unique 65 232 "Create Separate Link File?** If "no,** then the link 

id (e.g.±l res=:100. Polish Arm). Then any word, sentence, description is added immedtatdy following the tag defining 

section, image, or hot-spot in an image can be linked to this the image in the IFF source file, 234 "Add Link Description 
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Immediately Following l^g De&ung The Image In IFF hand, it is believed essential for all operatcrs and technicians 

Source File." Processing is then complete, 240 ''End,*' to have access to the training, certification, and documen- 

If a separate link file is desiied, then an ASCII Text Editor tation applications on the workstations dedicated to their 

(comm^dally available) is employed to create a hyper- own workplace, rather than on some remotely-located woik- 

graphic link file, with an extension of an *Jigl, 236 **Use s station. Therefore, alternative methods for generating and 

ASCn Text Editor To Create Hypergraphic Link File" All playing motion video and animation are desired, 
hot spots for the image are defined with one link line per hot The resulting package, referred to herein as '*Poor Man's 

spot Thereafter, the link file is specified in &e IFF source \^deo*' (PMV), can be written in C and PM, and can be 

file in conjunction with :artwork tag. 238 ''Specify The Link accessed from training or certification documents as an 

File In IFF Source File In Conjunction With Image, Tag." lO application-controlled window, as shown below: 
When separately defined, processing is complete, 240 

♦^d.« 

Cocffdinating text, graphics and images in a multimedia "*=^ id=vidfio x=cettter y^^ wm^^AOOx height=30Qi 

environment is very important to effective presentation and S^^^^^^^™^*"^ ^ 

management of the subject matter. Some researchers have 15 :acrrtcwportdU=*pmv*objectnamc»'PmvMam' Objoctiii£(^*npid.plb' 

investigated the automatic generation of coordinated multi- objectid=i. 
media. The methodology proposed ho-ein is based on a 

pre-defined presentation scenario and is implemented in one pj^y allows flic user to specify a set of OS/2 bitmaps as 

of the pre-defined tenq)lates (FIGS. 16a-16p). Multimedia y^^^ f^j^^^^ ^ (1-99999 ms) 

documents are partitioned into 'logical Pages". A logical 20 ^ ^^^^ delays for each frame or a group of frames 
page (FIG. 17) consists of a set of images and aU their 25). About 45 frames per second (256 color 300x400 

associated textual and audio description. In other words, a pj^^ls frame) can be achieved on an IBM Model 95 PS/2 

logical page is tiie pre-defined coUection of aU related (486DX2) with 32 MB RAM, with an initial load time of 7 

information (text, graphics, images, audio, animation, video seconds. The PMV's graphical front-end (FIG. 25) aUows 

dips, . . . ) which die trainee should/could review when 23 application developers to create video and animation clips 

studying the subject matter covered in ttiat page. The ^^^^y ^denUy. 

appearance, format, and access to logical pages are fixed and pj^ ^an generate eiflicr a "run-time'* PLB (PMV Library 

consistent throughout flie system Logical pages are related ^^^^^ ^ ^ "compile-time" PLB. ConmQe-time PLBs arc 

to eadi otfier via hypertext and hypergraphic links. To self-contained and can be exported to other workstations 

accommodate aU possible situations and to mamtam con- 30 withoutany need for copying the images or having access to 
sistency among various multimedia modules developed in a Run-time PLBs, on the otho" hand, contain pointers to 

organization, the Hbrary of 'logical Page Templates" is the images used in the video clip or animation and therefore 

px)vided (FIGS. I6a~16p). require access to these. In some cases, images are used in 

Many users of a multimedia training system sudi as jn^e than one animation or video cUp. Run-time PLBs are 

described herein will not be sophisticated computer users, 35 j^,^^ ^ffident in sudi cases since these are generated and 

and too many features (options, menus, controls) could be leaded at run-time. Thus, a single copy of flie frames to be 

confusmg, and m some cases restrict flie amount of control used in animation or video dip only needs to be stored, 

and access tfiat CMKB modules need to preserve. For When an animation or a video dip is launched, tiie run-time 
example, if the users are allowed to minimize training j^^ds flie images on flie fly, which could be time 

applications, flic system would not be able to control flie 40 consuming for long video dips (about 1 second per frame 

number of copies opened and would not know wheflier a ^Up on a 486 PS/2 witti 32 Meg RAM). In addition, tiie 
task has been completed ^^^^ f^^. run-time PLBs if flie images cannot be 

In addition to flie logical page templates dcscnbed above. located (moved or renamed). On flie otiier hand, fliere is flie 

most of flie major attributes of a logical page can be storage ovcriiead associated wifli compile-time PLBs since 
customized (HG. 19). For example, page background color, 45 ^ave to be repUcated and stared locally. Accessing 

hi^ght color, logical page sue and posiUon, text style and pj^g network is slow and unpredictable, 

font size, puU down menus and menu items, system menu PMV has two main portions, namely, a conq)flc/gencrate 

Items, push buttons, vertical and horizontal scrolls, min/max ^ideo and a play video function. There are four conmonents 
icons window tittes, etc. implement fliese functions. These are: 

Multimedia m general, and motion video, animation, and 50 pMBC.exe-^compiler 

audio in particular, are believed to be appealing to manu- PMV.DLL->playcr 

facturing personnel involved wifli maintenance, diagnostics, ♦.PMV-^video script file CTable 1) 

and training. HowevM-, in many cases the cost and logistics « PLB->video library block 

of required hardware and software is a major concern. yj^^ ^ 3^ ^ ^ 

Ranember thatiMSt manufacW^^ 55 defined miiuaUy via any text/ASCn editor or autoLdcaUy 

and even hundreds of workstations^ IVmning^ certification ^ ^ j^^^^ ^ ^^^^ ^^^^ 

and on-line document have to be mstaUed on a latge ^y way of exaniple, PMV video scdpt language fonnat is 

number of these workstations. Ji an appucation requires defined in Table 1 bdow 
special hardware or software license, the cost would become 

a significant roadblock. Alfliough cost is a major concern, 60 tart r i 

another problem might be the lack of any available expan- — — — 

sion slot on some workstations in the area. * This pmv file contains tie description of tbe video dip you aze 

It is not practical to expect or justify upgrading all creating. 

, ^ . .. . V -m**! ,T ^ • Each line has 5 fields 

workstations wifli a video card (e.g., IBM*s M-motion • TTc™* nTm^rr a. c ^ oxc j i. • 

, , , X , ^ - « , . t) Frame BMP fall path, 2) a maque frame id, 3) frame dehy m ms 

adapter card for moUon video), \^dco Capture Adapter (for 65 ♦ 4) Frame groap id Cif m)> 5) Group # of lepetition 

digitizing and storing of confessed video images), and flie * Complete this file then compile it to czeate adLpIb. 

Audio Capture and Playback Adapter (ACPA). On tfie other 
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* pDirc adl^mrr cApnTjectsSadlUdLpIb 

* adlplb is used in the ipf as sbown below: 

* racvicwport dll=TMV 6b)ectvicw='Pnn^if!aiii* 
objectinfo='fldIplb' 

•objectkfcl 

* play dkc fioUowii^ frames for 300 ms 
t^ft^tct^adl\adnJbaxp 1 300 0 0 
ei^itQ9ect^\adAadl2.biiq» 2 300 0 0 
e:^rrqtectsAad]\KU3.bnip 3 300 0 0 
eApioject^WU\adl4.biiip 4 300 0 0 

• 

« define and repeat ^ group #1* 3 times, far 200 ms 
cApioicct^UdIVadl3.binp 1 200 1 3 
ci^>io|ect^\adI\adM.fanq> 2 200 1 3 
e^iiDiect^UdI\adl3.bmp 3 200 1 3 
ciin^tct^ad^iidi4Jbmp 4 200 1 3 

* define and repeat the group #2, 3 dmes, for 100 ms 
eApfqjects\adAadl3isnp 1 100 2 3 
eApccjcct^ttd]\aril4.bmp 2 100 2 3 
e:^roject^ari1\adn hmp 3 100 2 3 
e:^xcgect^bdAadl4Jnip 4 100 2 3 

* 

* define and x^at die group #3, 3 times, for 50 ms 
e^;Hpjects\adKadl3.bn9 1 SO 3 3 
e^>rqjects\adIV«n4.fanq;> 2 SO 3 3 
ef^ectiUdnad]3.biiq> 3 SO 3 3 
c:^ttoject5\adI\adl4.hmp 4 SO 3 3 

* 

* define and repeat the group #4, 20 times, for 5 ms 
«e^iect^adI\adl3.bDq> 3 S 4 20 
^.'^rici^tAadlbdH.biiip 4 S 4 20 

« 

* define and repeat the group #5, 20 times, fisr SO ms 
ei)rt9ects\adIUdl3.baQ> 3 SO S 20 
e:Fror)cct!AadI\adl4.famp4 SO 5 20 

* 

* (fefinc and repeat the group #6, 20 times, for ISO ms 
eApiqiects\adAadI3.bnqi 3 130 6 20 
e:\Fcqiccts\adKadI4.boip 4 130 6 20 
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FIG. 26 depicts the process in greater detail As noted, a 
user employs either an ASCII editor or the PMV fi'ontrend 
300 to define a PMV saipt file 302, referred to herein as 
♦.mv (See Table 1). 

The compiler, PMVCexe 304 then takes the PMV script 
file and con^>iles it iato either a compile-time video library 
block 306 or a run-time video library block 308. The player. 
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then the bit m^ are appended to the bottom of the *PLB 
file directly, 336 "Append Bit Maps To Bottom Of ♦ J»LB " 
Alternatively, if nm-dme is desired, then the bit map 
addresses are af^nded to the video library block, 330 
"i^ipend AH BMP Addresses To *PLB " Compilation is 
then comply, 340 *^d." 

One processing anbodimenl of a PMV player is depicted 
in FIG. 28. As shown, (he player, 350 TM^DLL," initially 
reads the video library block header and obtains the bit map 
pointers and delay parameters, 352 **Rcad The *J*LB 
Header, Bit M^ Pointers And Delay Parameters." Next, the 
player determines lA^ether the file oonqirises compile-time 
or run-time information, 354 Xon^ile-Ume?" If compile- 
time, the Mt maps appended to the video library block are 
read directly according to fiie bit map pointers and loaded 
into memory, 356 '"Read Bit M^s fended To ♦.PLB File 
According To Pointers." If run-time, then the hit map 
addresses are read from the video hlHary block and loaded 
into memory, 358 **Read Bit Map Addresses Listed In 
*.FLB." 

Processing next determines wbetiier compression of the 
bit maps has occurred, 360 "Conc^ession?** If **ycs," then 
die bit m2^ are deconqxressed, 362 "Decompress Bit 
Maps." Thereafter, the bit maps are loaded into memoiy, 364 
**Load Bit Maps Into Memory." 

The header information is neKt converted into bit map 
pointers and bit map ddays, 366 '*Conv^ The Header Bit 
M^ Ofifeet Pointers To The Memory Offsets Of The Bit 
M^s." The player then iterates through the video display 
list displaying the bit maps for the specified time, 368 
"Iterate Through Ust And Display Bit Maps For The Time 
Specified In Corresponding Delay Parameter.'* The process 
is continued untQ the user or the application closes the video 
window Off the logical page, 370 "Video Window Or Logical 
Page Qosedr If **yes,** playing is complete, 372 ''End." 

PMV can be used f<x pseudo animation using graphic bit 
maps or still images with SCO ms to 1 second of firame delay. 
Alternatively, a true animation tool from IBM called AnDes 
(Animated Design) can be used, which allows animated 
models to be aeated using Bxcelerator, marketed by Intcr- 
solv Inc. of Rockwell, Md., based on Ward & Mellor 
methodology Parameswaran, T., '^Structured Design of Real 
Time Systems," Proc, of National Workshop on Software 
Reliability EngLoeedng, WSRE92, (1992), which is based 



PMVDLL 310, then retrieves either the compile-tiine library 

or the nm-time library, and repeats each fi-ame by the global 45 on transfonnation gr^hs and state transition diagrams. 



delay, c.g., 300 msec, Le., unless a specific frame delay is 
spedfied otherwise. The player presents the PMV within a 
video window in a logical page 312. Detailed embodiments 
of the above-described processing are presented in FIGS. 
27-28. 

Referring fint to HG. 27, Ae con^ilff, 320 "YMVCexe," 
mitiaUy reads the ASCII video script file, 322, 'Tlead ♦PMV 
Script File." A unique identifier is then assigned to each bit 
m^ based on order of ^earance, 324 *list And Tag All Bit 



Excelerator models are then converted to AnDes format, 
before they can be animated. Although AnDes offers a true 
animation capability, in most cases use of PMV is believed 
preferable to avoid the ovcihead associated with installing 
50 Excelerator, AnDes, and Prolog for creating and performing 
animation. 

Atthougjh specific embodiments of the present invention 
have been illustrated in the accompanying drawings and 
described in the foregoing detailed description, it will be 



Maps In Order Of ;^)pcarancc." The video script is then ss understood that the invention is not limited to the particular 



flatten and saved as a header in the video library block 
♦.PLB in addition to loading the apprc^niatc bit maps into 
the library or ttie bit m^ addresses depending upon whether 
coiiq)ile-time orrun-tiine is desired, 326 *Tlatten Script And 
Save As Headw Of Gen^ate Pointers To Bit Mq)s." 

Next, the conquer determines whether run-time or 
conq)ilc-time is desired, 328 "Con^ile-Time?" If **yes," 
then inquiry is made whether compression is needed, 332 
'^Compression?" If again "yes," each bit map is conqiressed 
and appended to the bottom of the play file '.PLB, 334 
'•Compress Each Bit Map And Append To The Bottom Of 
*.FLB Wth No Spacing." If compression is unnecessary. 
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embodiments described herein, but is capable of numerous 
rearrangements, modifications and substitutions without 
departing horn the scope of the invention. The following 
claims are intended to encon^ass all such modifications. 
I claim: 

1. An e^>^ system responsive to provided external 
information, said expert system conqidsing: 

first memory means for staring a hierarchically-structured 
knowledge base containing inferentially processable 
information, wherein said hierarchically-structured 
knowledge base conqxrises a hierarducal decomposi* 
tion comprising a plurality (^knowledge trees coupled 
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together in multiple levels, said multiple levels being 
arranged accarding to attributes associated with said 
plurality of knowledge frees, and whereia said multiple 
levels comprise three distinct levels, said three distinct 
levels comprising an uppermost level having a domain 
behavioral knowledge tree, a middle level having a 
domain stmctural knowledge tree and a lowennost 
level having a domain action tree; 

second memory means for storing at least one miiltfmfidia 
or hypermedia utility containing non-inferentially 
accessible information; and 

processing means coupled to said first memory means and 
to said second memory means for processing said 
hierarchically-structured knowledge base and for 
accessing said at least one utiHty, respectively, said 
processing means con:q)rising means for inferential 
processing of information in said hio'archically- 
structured knowledge base and means for non- 
inferential accessing of information in said at least one 
utility, said means for inferential processing of infor- 
mation and said means for non-inferential accessing of ^ 
information being responsive to the provided external 
information, and wherein said three distinct levels of 
said hiaardiically-stiuctured knowledge base arc inter- 
connected such that said means for inferential process- 
ing of information inference processes throu^ said 
hierarchically-structured knowledge base from said 
domain behavioral knowledge level through said 
domain structural knowledge level to said domain 
action level. 

2. The expert system of daim 1, wherein said processing 
means is domain independent 

3. The expert system of claim 1, wherein said 
hierarchically-structured knowledge base comprises a diag- 
nostic knowledge base and wherein said means for inferen- 
tial processing of information in said hierarchlcally- 
stractured knowledge base proceeds interactively with 
external information provided by a user of the expert system. 

4. The expert system of claim 1, wherein said means for 
inferential processing of informatibn in said hierarchically- 



10. The expert system of daim 7, wherein said expert 
system comprises a multimedia/hypomcdia expat system, 
and wherein said at least one utility contains non- 
infercntially accessible, mnltimedia displayable information 
and hypermedia displayable infonnatioiL 

11. The expat system of daim 10, wherein said hyper- 
media displayable information has a predefined gr^hic 
image that when selected by a user provides an options menu 
having at least two user selectable options, eadi user sdect- 
able option of said at least two user sdectable options being 
linked to a separate portion of the multimedia displayable 
information, each separate portion of said multimedia dis- 
playable information presenting information to the user of 
the expert system in a different multimedia format upon user 
selection of the linked uso- selectable option. 

12. The expert system of daim 10, further comprising 
means for presenting said multimedia displayable informa- 
tion and said hypomedia displayable information to the user 
of the expert system using fixeil presentation templates. 

13. The expert system of claim 1, fiirther comprising a 
display means and a graphical user interface, said graphical 
user interface being coupled to said display means and to 
said processing means for coordinating a user's interactive 
providing of external information to the expert system- 

14. The expert system of claim 13, wherein said expert 
system comprises a multilevel training and certification 
system and wherein said at least one utility indudes an 
active training module for guiding and monitoring a user's 
training. 

15. The expert system of daim 1, farther comprising an 
external application utility, and wherein said processing 
means is coupled such that said means for non-inferential 
accessing of information includes means for non-inferential 
accessing of information from said external application 

35 utility. 

16. The expert system of daim 1, further in combination 
with a supervised knowledge acquisition means for prede- 
fining said hierarchically-structured knowledge base. 

17. An intelligent system responsive to provided external 
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Structured knowledge base indudes means for en^)loying 40 information, said intelligent system con^ffising: 



said means for non-inferential accessing of information in 
said at least one utility during inferential processing of 
information in said hierarchically-structured knowledge 
base, 

5. The expert system of daim 1, wherein at least some 45 
nodes in the hierarducal decomposition have entropy values 
associated therewith, each entropy value being related to at 
least a node strengtii or a node cost for the assodated node, 
wherein said means for inferential processing of information 

of said hierarchically-structured knowledge base indudes 
means for employing the entropy values to infcrentially 
process information. 

6. The expert system of claim 1, further in combination 
with a supervised knowledge acquisition means for prede- 
fining said hierarchically-stnictured knowledge base. 

7. The expert system of claim 1, wherein said at least one 
utility comprises one of a training module, an on-line 
documentation module, a sorice module, and a certification 
module. 

8. The expert system of daim 7, wherein said expert €o 
system comprises a multimedia expert system and wherein 
said at least one utility contains non-inferentially accessible, 
multimedia displayable informadon. 

9. The expert system of daim 7, wherein said expert 
system comprises a hypermedia expert system and herein 65 
said at least one utility contains non-inf erendally accessible, 
hypermedia displayable information. 
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memory means for storing multimedia displayable 
information, said multimedia displayable information 
being non-inferentiaUy accessible; and 
an intelligent control system coupled to said memory 
means, said intelligent control system induding a 
hierarchically-structured knowledge base containing 
infcrentially processable information and processing 
means coupled to said hierarchically-structured knowl- 
edge base, wherein said hierarchically-structured 
knowledge base comprises a hierarchical decom^si- 
tion comprising a plurality of knowledge trees coupled 
together in mult^le levels, said multiple levels being 
arranged according to attributes assodated with said 
plurality of knowledge trees, wherein said multiple 
levels con^rise three distinct levds, said three distinct 
levels comprising an uppermost level having a domain 
behavioral knowledge tree, a middle levd having a 
domain structural knowledge tree, and a lowennost 
level having a domain action tree, said processing 
means induding means for inferential processing of 
information in said hierarchically-structured knowl- 
edge base and means for non-inferential accessing of 
said multimedia displayable information stored in said 
memory means, said means for non-inferential access- 
ing of said multimedia displayable information being 
responsive to said means for inferential processing of 
information, and said means for inferential processing 
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of infonnatton being responsive to the provided extsx- separate portion of &e multimedia displayable infonnatioD 

nal inf<Hmation, and wherein said three distinct levels stored in the memoiy means. 

of said hierarchically-structured knowledge base arc 20. Hie intelligent system of claim 19, wherein each 

interconnected such that said means for inferential separate portion of said multimedia displayable information 

processing of infonnalion inference processes througji 5 presents information in a diffocnt multimicdia format to the 

said hierarduc£dly-stru(^ taiowledge base from intelligent syst^ via said display means upon 

said dom^ behavioral knowledge level through said ^^^^^ of the linked user selectable option, 

J^lev'S ^^^1^ 21. THe intelligent system of daim 20, wherein said 

la^T^e hSEdligent system of claim 17, wherein said lO dif crcnt imdtimcdia f orma^ 

-,«,«TftJ^fK«. o/^««ric*c. cal image format, and a video/animation format 

nieinory means nirtber comprises: *^\2r . «, « . . ^ 

r . ' X. J. 1 t_i • * 22. The intelligent system of claim 17, ftmhercompnsmg 

means for storing hypermedia displayable information, ^ „c^t^*^^ ^«„«u.i ;«t*ii;™7To«#TvS 

said hypermema di^layable i^ormation also being ^ graphical usa: mterfacc coupled to said mtelhgent confrol 

non-infSlly acccssiTblc; and fadhtatmg user providmg of said external infor- 

i< mation. 

wherein said intcUigent control system's means for non- ^ intelligent system of claim 22, wherein said 

^rSSS"^ ^Sh^^^^^h" control systcrn includes user controllable means 

able infStiors31n said me^^^s ^" creating and altering the multimedia displayable infar- 

19. The intelligent system of claim 18, wherein said "^^l!^ T^^."" ^^^''^ ^ u • . 

intelligent control system includes display means, and 24. The mtcUigcnt system of daim 22, whcrem said user 

wherein said hypennedia displayable information includes a conttoUable means for aeating and altering multimedia 

predefined gr^hics image displayable on said display displayable information in video or animation format 

means, said processing means of said intelligent control includes user selectable means for establishing a run-time 

system inchjding means fcs providing the user of the Intel- video library or a compile-time video library prior to display 

ligent system with an options menu upon user selection of ^ of said multimedia displayable information in video or 

the predefined gr^hics image, said options menu having at animation format 
least two user selectable options, eadi user selectable option 

of said at least two user selectable options being linked to a ***** 
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